Dealing with errors after a run#

In this example, we run the model on a list of three glaciers: two of them will end with errors: one because it already failed at preprocessing (i.e. prior to this run), and one during the run. We show how to analyze theses erros and solve (some) of them, as described in the OGGM documentation under troubleshooting.

Continue on error#

# Locals
import oggm.cfg as cfg
from oggm import utils, workflow, tasks, DEFAULT_BASE_URL

# Libs
import os
import xarray as xr
import pandas as pd

# Initialize OGGM and set up the default run parameters
cfg.initialize(logging_level='WARNING')

# Here we override some of the default parameters
# How many grid points around the glacier?
# We make it small because we want the model to error because
# of flowing out of the domain
cfg.PARAMS['border'] = 80

# This is useful since we have three glaciers
cfg.PARAMS['use_multiprocessing'] = True

# This is the important bit!
# We tell OGGM to continue despite of errors
cfg.PARAMS['continue_on_error'] = True

# Local working directory (where OGGM will write its output)
WORKING_DIR = utils.gettempdir('OGGM_Errors', reset=True)
cfg.PATHS['working_dir'] = WORKING_DIR

rgi_ids = ['RGI60-11.00897', 'RGI60-11.01450', 'RGI60-11.03235']

# Go - get the pre-processed glacier directories
# in OGGM v1.6 you have to explicitly indicate the url from where you want to start from
# we will use here the elevation band flowlines which are much simpler than the centerlines
gdirs = workflow.init_glacier_directories(rgi_ids, from_prepro_level=5, prepro_base_url=DEFAULT_BASE_URL)

# We can step directly to the experiment!
# Random climate representative for the recent climate (1985-2015)
# with a negative bias added to the random temperature series:
workflow.execute_entity_task(tasks.run_random_climate, gdirs, y0=2000,
                             nyears=150, seed=0,
                             temperature_bias=-2)
2024-02-02 16:58:08: oggm.cfg: Reading default parameters from the OGGM `params.cfg` configuration file.
2024-02-02 16:58:08: oggm.cfg: Multiprocessing switched OFF according to the parameter file.
2024-02-02 16:58:08: oggm.cfg: Multiprocessing: using all available processors (N=4)
2024-02-02 16:58:08: oggm.cfg: Multiprocessing switched ON after user settings.
2024-02-02 16:58:08: oggm.cfg: PARAMS['continue_on_error'] changed from `False` to `True`.
2024-02-02 16:58:09: oggm.workflow: init_glacier_directories from prepro level 5 on 3 glaciers.
2024-02-02 16:58:09: oggm.workflow: Execute entity tasks [gdir_from_prepro] on 3 glaciers
2024-02-02 16:58:09: oggm.workflow: Execute entity tasks [run_random_climate] on 3 glaciers
2024-02-02 16:58:09: oggm.core.flowline: InvalidWorkflowError occurred during task run_random_climate on RGI60-11.03235: Need a valid `model_flowlines` file. If you explicitly want to use `inversion_flowlines`, set use_inversion_flowlines=True.
2024-02-02 16:58:10: oggm.core.flowline: RuntimeError occurred during task flowline_model_run on RGI60-11.00897: Glacier exceeds domain boundaries, at year: 119.83333333333333
[None, <oggm.core.flowline.SemiImplicitModel at 0x7fbe2938ba50>, None]

Error diagnostics#

# Write the compiled output
utils.compile_glacier_statistics(gdirs);  # saved as glacier_statistics.csv in the WORKING_DIR folder
utils.compile_run_output(gdirs);  # saved as run_output.nc in the WORKING_DIR folder
2024-02-02 16:58:11: oggm.utils: Applying global task compile_glacier_statistics on 3 glaciers
2024-02-02 16:58:11: oggm.workflow: Execute entity tasks [glacier_statistics] on 3 glaciers
2024-02-02 16:58:12: oggm.utils: Applying global task compile_run_output on 3 glaciers
2024-02-02 16:58:12: oggm.utils: Applying compile_run_output on 3 gdirs.
# Read it
with xr.open_dataset(os.path.join(WORKING_DIR, 'run_output.nc')) as ds:
    ds = ds.load()
df_stats = pd.read_csv(os.path.join(WORKING_DIR, 'glacier_statistics.csv'), index_col=0)
# all possible statistics about the glaciers
df_stats
rgi_region rgi_subregion name cenlon cenlat rgi_area_km2 rgi_year glacier_type terminus_type is_tidewater ... melt_f prcp_fac temp_bias reference_mb reference_mb_err reference_period temp_default_gradient temp_all_solid temp_all_liq temp_melt
rgi_id
RGI60-11.00897 11 11-01 Hintereisferner 10.75840 46.8003 8.036 2003 Glacier Land-terminating False ... 4.921376 3.533170 1.758313 -1100.3 171.8 2000-01-01_2020-01-01 -0.0065 0.0 2.0 -1.0
RGI60-11.01450 11 11-01 NaN 8.01919 46.5028 82.206 2003 Glacier Land-terminating False ... 6.364902 1.421212 0.920699 -1210.7 131.5 2000-01-01_2020-01-01 -0.0065 0.0 2.0 -1.0
RGI60-11.03235 11 11-01 At4J131Ga001 Eiskar 12.90700 46.6120 0.151 1973 Glacier Land-terminating False ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN

3 rows × 58 columns

  • in the column error_task, we can see whether an error occurred, and if yes during which task

  • error_msg describes the actual error message

df_stats[['error_task', 'error_msg']]
error_task error_msg
rgi_id
RGI60-11.00897 flowline_model_run RuntimeError: Glacier exceeds domain boundarie...
RGI60-11.01450 NaN NaN
RGI60-11.03235 simple_glacier_masks GeometryError: RGI60-11.03235 is a nominal gla...

We can also check which glacier failed at which task by using compile_task_log.

# also saved as task_log.csv in the WORKING_DIR folder - "append=False" replaces the existing one
utils.compile_task_log(gdirs, task_names=['glacier_masks', 'compute_centerlines', 'flowline_model_run'], append=False)
2024-02-02 16:58:13: oggm.utils: Applying global task compile_task_log on 3 glaciers
glacier_masks compute_centerlines flowline_model_run
rgi_id
RGI60-11.00897 RuntimeError: Glacier exceeds domain boundarie...
RGI60-11.01450 SUCCESS
RGI60-11.03235

Error solving#

Glacier exceeds domain boundaries#

To remove this error just increase the domain boundary before running init_glacier_directories ! Attention, this means that more data has to be downloaded and the run takes more time. The available preprocessed directories for cfg.PARAMS['border'] for OGGM v1.6 are 10, 80, 160 or 240 at the moment; the unit is number of grid points outside the glacier boundaries. More about that in the OGGM documentation under preprocessed files.

# reset to recompute statistics
# Beware! If you use `reset=True` in `utils.mkdir`, ALL DATA in this folder will be deleted!
utils.mkdir(WORKING_DIR, reset=True)

# increase the amount of gridpoints outside the glacier
cfg.PARAMS['border'] = 160
gdirs = workflow.init_glacier_directories(rgi_ids, from_prepro_level=5, prepro_base_url=DEFAULT_BASE_URL)
workflow.execute_entity_task(tasks.run_random_climate, gdirs, y0=2000,
                             nyears=150, seed=0,
                             temperature_bias=-2);

# recompute the output
# we can also get the run output directly from the methods
df_stats = utils.compile_glacier_statistics(gdirs)
ds = utils.compile_run_output(gdirs)
2024-02-02 16:58:13: oggm.cfg: PARAMS['border'] changed from `80` to `160`.
2024-02-02 16:58:13: oggm.workflow: init_glacier_directories from prepro level 5 on 3 glaciers.
2024-02-02 16:58:13: oggm.workflow: Execute entity tasks [gdir_from_prepro] on 3 glaciers
2024-02-02 16:58:14: oggm.workflow: Execute entity tasks [run_random_climate] on 3 glaciers
2024-02-02 16:58:14: oggm.core.flowline: InvalidWorkflowError occurred during task run_random_climate on RGI60-11.03235: Need a valid `model_flowlines` file. If you explicitly want to use `inversion_flowlines`, set use_inversion_flowlines=True.
2024-02-02 16:58:16: oggm.utils: Applying global task compile_glacier_statistics on 3 glaciers
2024-02-02 16:58:16: oggm.workflow: Execute entity tasks [glacier_statistics] on 3 glaciers
2024-02-02 16:58:16: oggm.utils: Applying global task compile_run_output on 3 glaciers
2024-02-02 16:58:16: oggm.utils: Applying compile_run_output on 3 gdirs.
# check again
df_stats[['error_task', 'error_msg']]
error_task error_msg
rgi_id
RGI60-11.00897 None None
RGI60-11.01450 None None
RGI60-11.03235 simple_glacier_masks GeometryError: RGI60-11.03235 is a nominal gla...

Now RGI60-11.00897 runs without errors!

RGI60-11.xxxxx is a nominal glacier.#

This error message in the log is misleading: it does not really describe the source of the error, which happened earlier in the processing chain. Therefore we can look instead into the glacier_statistics via compile_glacier_statistics or into the log output via compile_task_log:

print('error_task: {}, error_msg: {}'.format(df_stats.loc['RGI60-11.03235']['error_task'],
                                             df_stats.loc['RGI60-11.03235']['error_msg']))
error_task: simple_glacier_masks, error_msg: GeometryError: RGI60-11.03235 is a nominal glacier.

Now we have a better understanding of the error:

  • OGGM can not work with this geometry of this glacier and could therefore not make a gridded mask of the glacier outlines.

  • there is no way to prevent this except you find a better way to pre-process the geometry of this glacier

  • these glaciers have to be ignored! Less than 0.5% of glacier area globally have errors during the geometry processing or failures in computing certain topographical properties by e.g. invalid DEM, see Sect. 4.2 Invalid Glaciers of the OGGM paper (Maussion et al., 2019) and this tutorial for more up-to-date numbers

Ignoring those glaciers with errors that we can’t solve#

In the run_output, you can for example just use *.dropna to remove these. For other applications (e.g. quantitative mass change evaluation), more will be needed (not available yet in the OGGM codebase):

ds.dropna(dim='rgi_id')  # here we can e.g. find the volume evolution
<xarray.Dataset>
Dimensions:         (time: 151, rgi_id: 2)
Coordinates:
  * time            (time) float64 0.0 1.0 2.0 3.0 ... 147.0 148.0 149.0 150.0
  * rgi_id          (rgi_id) <U14 'RGI60-11.00897' 'RGI60-11.01450'
    hydro_year      (time) int64 0 1 2 3 4 5 6 7 ... 144 145 146 147 148 149 150
    hydro_month     (time) int64 4 4 4 4 4 4 4 4 4 4 4 ... 4 4 4 4 4 4 4 4 4 4 4
    calendar_year   (time) int64 0 1 2 3 4 5 6 7 ... 144 145 146 147 148 149 150
    calendar_month  (time) int64 1 1 1 1 1 1 1 1 1 1 1 ... 1 1 1 1 1 1 1 1 1 1 1
Data variables:
    volume          (time, rgi_id) float64 6.154e+08 1.206e+10 ... 1.652e+10
    volume_bsl      (time, rgi_id) float64 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 0.0
    volume_bwl      (time, rgi_id) float64 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 0.0
    area            (time, rgi_id) float64 8.036e+06 8.221e+07 ... 9.464e+07
    length          (time, rgi_id) float64 5.6e+03 1.891e+04 ... 2.877e+04
    calving         (time, rgi_id) float64 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 0.0
    calving_rate    (time, rgi_id) float64 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 0.0
    water_level     (rgi_id) float64 0.0 0.0
    glen_a          (rgi_id) float64 6.007e-24 5.939e-24
    fs              (rgi_id) float64 0.0 0.0
Attributes:
    description:    OGGM model output
    oggm_version:   1.6.2.dev23+g5f8a8c5
    calendar:       365-day no leap
    creation_date:  2024-02-02 16:58:16

What’s next?#