Data & I/O

Data & I/O#

Data formats#

The model is data-driven, with all inputs defined in a collection of CSV files located in a dedicated case directory. These files define the model’s sets (e.g., periods, technologies, nodes) and parameters (e.g., costs, capacities, efficiencies).

CSV File Naming Conventions#

The input CSV files follow a specific naming convention to distinguish between different types of data:

  • ``oM_Dict…``: Files starting with this prefix are used to define the model’s sets and dictionaries. These typically contain lists of technologies, nodes, or other categorical data.

  • ``oM_Data…``: Files starting with this prefix contain the numerical data for the model’s parameters, such as costs, efficiencies, or time series data.

This convention helps in organizing the input data and is used by the data loading functions to correctly process the files.

CSV File Descriptions#

The following sections provide a more detailed description of the oM_Dict... and oM_Data... CSV files.

oM_Dict Files#

These files define the sets and dictionaries used in the optimization model. Each file corresponds to a specific set, and the rows of the CSV file define the elements of that set.

oM_Dict Files#

File

Description

oM_Dict_Node_Home1.csv

Defines the nodes in the network. Each row represents a node. The relevant column is Node.

oM_Dict_Technology_Home1.csv

Defines the technologies available in the model. Each row represents a technology. The relevant column is Technology.

oM_Dict_Area_Home1.csv

Defines the geographical areas. The relevant column is Area.

oM_Dict_Region_Home1.csv

Defines the geographical regions. The relevant column is Region.

oM_Dict_Zone_Home1.csv

Defines the geographical zones. The relevant column is Zone.

oM_Dict_Period_Home1.csv

Defines the time periods. The relevant column is Period.

oM_Dict_Scenario_Home1.csv

Defines the scenarios. The relevant column is Scenario.

oM_Dict_LoadLevel_Home1.csv

Defines the load levels. The relevant column is LoadLevel.

oM_Dict_Storage_Home1.csv

Defines the storage technologies. The relevant column is Storage.

oM_Dict_Circuit_Home1.csv

Defines the electrical circuits. The relevant column is Circuit.

oM_Dict_ElectricityDemand_Home1.csv

Defines the electricity demand categories. The relevant column is ElectricityDemand.

oM_Dict_ElectricityGeneration_Home1.csv

Defines the electricity generation categories. The relevant column is ElectricityGeneration.

oM_Dict_ElectricityRetail_Home1.csv

Defines the electricity retail categories. The relevant column is ElectricityRetail.

oM_Dict_HydrogenDemand_Home1.csv

Defines the hydrogen demand categories. The relevant column is HydrogenDemand.

oM_Dict_HydrogenGeneration_Home1.csv

Defines the hydrogen generation categories. The relevant column is HydrogenGeneration.

oM_Dict_HydrogenRetail_Home1.csv

Defines the hydrogen retail categories. The relevant column is HydrogenRetail.

oM_Dict_AreaToRegion_Home1.csv

Maps areas to regions. The relevant columns are Area and Region.

oM_Dict_ZoneToArea_Home1.csv

Maps zones to areas. The relevant columns are Zone and Area.

oM_Dict_NodeToZone_Home1.csv

Maps nodes to zones. The relevant columns are Node and Zone.

oM_Data Files#

These files contain the numerical data for the model’s parameters. This includes time series data, costs, efficiencies, and other parameters that define the behavior of the model.

oM_Data Files#

File

Description

oM_Data_Duration_Home1.csv

Specifies the duration of each time step.

oM_Data_ElectricityDemand_Home1.csv

Time series data for electricity demand. Columns: Period, Scenario, LoadLevel, value.

oM_Data_ElectricityGeneration_Home1.csv

Parameters for electricity generation technologies (e.g., capacity). Columns: Technology, parameter values.

oM_Data_ElectricityNetwork_Home1.csv

Data for the electricity network, like line capacities.

oM_Data_ElectricityRetail_Home1.csv

Data related to electricity retail.

oM_Data_HydrogenDemand_Home1.csv

Time series data for hydrogen demand.

oM_Data_HydrogenGeneration_Home1.csv

Parameters for hydrogen generation technologies.

oM_Data_HydrogenNetwork_Home1.csv

Data for the hydrogen network.

oM_Data_HydrogenRetail_Home1.csv

Data related to hydrogen retail.

oM_Data_NodeLocation_Home1.csv

Defines the geographical location (e.g., latitude, longitude) of nodes.

oM_Data_OperatingReserveActivation_Home1.csv

Cost or activation data for operating reserves.

oM_Data_OperatingReservePrice_Home1.csv

Prices for operating reserves.

oM_Data_OperatingReserveRequire_Home1.csv

Requirements for operating reserves.

oM_Data_Option_Home1.csv

Contains various modeling options and flags.

oM_Data_Parameter_Home1.csv

Defines global parameters like costs, discount rates, and time steps.

oM_Data_Period_Home1.csv

Defines the characteristics of each period, like duration.

oM_Data_Scenario_Home1.csv

Provides scenario-specific data, such as probabilities.

oM_Data_Tariff_Home1.csv

Contains electricity tariff data.

oM_Data_VarEnergyCost_Home1.csv

Variable costs associated with energy.

oM_Data_VarEnergyPrice_Home1.csv

Variable prices for energy.

oM_Data_VarMaxConsumption_Home1.csv

Upper bounds for consumption variables.

oM_Data_VarMaxDemand_Home1.csv

Upper bounds for demand variables.

oM_Data_VarMaxEmissionCost_Home1.csv

Upper bounds for emission cost variables.

oM_Data_VarMaxEnergy_Home1.csv

Upper bounds for energy variables.

oM_Data_VarMaxFuelCost_Home1.csv

Upper bounds for fuel cost variables.

oM_Data_VarMaxGeneration_Home1.csv

Upper bounds for generation variables.

oM_Data_VarMaxInflows_Home1.csv

Upper bounds for inflow variables.

oM_Data_VarMaxOutflows_Home1.csv

Upper bounds for outflow variables.

oM_Data_VarMaxStorage_Home1.csv

Upper bounds for storage variables (e.g., max capacity). Columns: Storage, Node, value.

oM_Data_VarMinConsumption_Home1.csv

Lower bounds for consumption variables.

oM_Data_VarMinDemand_Home1.csv

Lower bounds for demand variables.

oM_Data_VarMinEmissionCost_Home1.csv

Lower bounds for emission cost variables.

oM_Data_VarMinEnergy_Home1.csv

Lower bounds for energy variables.

oM_Data_VarMinFuelCost_Home1.csv

Lower bounds for fuel cost variables.

oM_Data_VarMinGeneration_Home1.csv

Lower bounds for generation variables.

oM_Data_VarMinInflows_Home1.csv

Lower bounds for inflow variables.

oM_Data_VarMinOutflows_Home1.csv

Lower bounds for outflow variables.

oM_Data_VarMinStorage_Home1.csv

Lower bounds for storage variables.

oM_Data_VarPositionConsumption_Home1.csv

Position data for consumption variables.

oM_Data_VarPositionGeneration_Home1.csv

Position data for generation variables.

oM_Data_VarPositionOutflows_Home1.csv

Position data for outflow variables.

oM_Data_VarShutDown_Home1.csv

Costs or parameters for shutting down units.

oM_Data_VarStartUp_Home1.csv

Costs or parameters for starting up units.

Loaders#

el1xr_opt.Modules.oM_LoadCase.load_case(directory=None, case=None, date=None, solver='highs', rawresults=False, plots=False, indlog=False)[source]

Create and validate a load case configuration.

Parameters:

  • dir (str, optional) – Directory where the case is located. Defaults to “../site-packages/el1xr_opt”.

  • case (str, optional) – Name of the case to load. Defaults to “Grid1”.

  • date (datetime or str, optional) – Date information. If string, it should follow “%Y-%m-%d %H:%M”. Defaults to current datetime (rounded to minute).

  • solver (str, optional) – Solver to be used. Defaults to “highs”.

  • rawresults (bool, optional) – Whether to save raw results. Defaults to False.

  • plots (bool, optional) – Whether to generate plots. Defaults to False.

  • indlog (bool, optional) – Whether to enable individual logging. Defaults to False.

  • directory (str | None)

Returns:

A dictionary containing the case configuration.

Return type:

dict

Writers#

el1xr_opt.Modules.oM_OutputData.save_to_csv(df, path, filename, index=False)[source]

Save a DataFrame to CSV file.

Parameters:

  • df (pd.DataFrame) – DataFrame to save

  • path (str) – Directory path

  • filename (str) – Filename including extension

  • index (bool) – Whether to include index in CSV

el1xr_opt.Modules.oM_OutputData.create_line_chart(df, x_col, y_col, color_col=None, title='', x_title='', y_title='', width=800, height=400, date_format='%a, %b %d, %H:%M', strokeDash=None)[source]

Create a reusable Altair line chart.

Parameters:

  • df (pd.DataFrame) – Data to plot

  • x_col (str) – Column name for x-axis

  • y_col (str) – Column name for y-axis

  • color_col (str, optional) – Column name for color encoding

  • title (str) – Chart title

  • x_title (str) – X-axis title

  • y_title (str) – Y-axis title

  • width (int) – Chart width

  • height (int) – Chart height

  • date_format (str) – Format string for date axis

  • strokeDash (list, optional) – Dash pattern for line [5, 5]

Returns:

Altair chart object

Return type:

alt.Chart

el1xr_opt.Modules.oM_OutputData.create_bar_chart(df, x_col, y_col, color_col, title='', x_title='', y_title='', width=800, height=400, date_format='%a, %b %d, %H:%M')[source]

Create a reusable Altair bar chart.

Parameters:

  • df (pd.DataFrame) – Data to plot

  • x_col (str) – Column name for x-axis

  • y_col (str) – Column name for y-axis (use ‘sum(…)’ for aggregation)

  • color_col (str) – Column name for color encoding

  • title (str) – Chart title

  • x_title (str) – X-axis title

  • y_title (str) – Y-axis title

  • width (int) – Chart width

  • height (int) – Chart height

  • date_format (str) – Format string for date axis

Returns:

Altair chart object

Return type:

alt.Chart

el1xr_opt.Modules.oM_OutputData.create_duration_curve(df, value_col, date_col, title, y_label, path, filename)[source]

Create and save a duration curve plot.

Parameters:

  • df (pd.DataFrame) – Input dataframe

  • value_col (str) – Column name containing values to plot

  • date_col (str) – Column name containing dates

  • title (str) – Chart title

  • y_label (str) – Y-axis label

  • path (str) – Directory path to save chart

  • filename (str) – Output filename

Returns:

DataFrame sorted by value (descending) with counter column

Return type:

pd.DataFrame

el1xr_opt.Modules.oM_OutputData.save_chart(chart, path, filename, embed_options=None)[source]

Save an Altair chart to HTML file.

Parameters:

  • chart – Altair chart object

  • path (str) – Directory path

  • filename (str) – Filename including extension

  • embed_options (dict, optional) – Options for embedding the chart. Defaults to {‘renderer’: ‘svg’}.

el1xr_opt.Modules.oM_OutputData.create_and_save_duration_curve(series_data, index_tuples, value_col_name, Date, hour_of_year, path, csv_filename, html_filename, title, y_label)[source]

Create and save a duration curve with both CSV and HTML outputs.

Parameters:

  • series_data (list) – List of values

  • index_tuples – Index tuples (e.g., from model.psn)

  • value_col_name (str) – Name for the value column

  • Date – Starting date

  • hour_of_year (str) – Hour of year reference

  • path (str) – Output directory

  • csv_filename (str) – CSV filename

  • html_filename (str) – HTML filename

  • title (str) – Chart title

  • y_label (str) – Y-axis label

Returns:

Processed dataframe with counter

Return type:

pd.DataFrame

el1xr_opt.Modules.oM_OutputData.saving_rawdata(DirName, CaseName, SolverName, model, optmodel, indlog)[source]

Save raw optimization model data to CSV files.

This function iterates through all active variables, parameters, and constraints in the optimization model and saves their data to separate CSV files.

  • Variables are saved with their values, lower bounds, and upper bounds.

  • Parameters are saved with their values.

  • Constraints are saved with their dual values.

Parameters:

  • DirName (str) – The directory where the result files will be saved.

  • CaseName (str) – The name of the case, used for subdirectory and file naming.

  • SolverName (str) – The name of the solver used.

  • model – The optimization model object.

  • optmodel – The concrete optimization model instance.

  • indlog – Logging indicator.

Returns:

The original optimization model object.

Return type:

model

el1xr_opt.Modules.oM_OutputData.saving_results(DirName, CaseName, Date, model, optmodel, indlog)[source]

Save processed optimization results to CSV files and generate plots.

This function processes the results from the optimization model to generate a series of CSV files and Altair plots for analysis. It covers:

  • Total costs (hourly and general)

  • Electricity balance (generation, consumption, flows)

  • Net and original electricity demand

  • State of energy for storage systems

  • Fixed availability of assets

  • A summary of all key output metrics

It also generates Sankey diagrams and duration curves for various metrics.

Parameters:

  • DirName (str) – The directory where the result files will be saved.

  • CaseName (str) – The name of the case, used for subdirectory and file naming.

  • Date (str or datetime) – The starting date for the results, used to calculate time-series data.

  • model – The optimization model object.

  • optmodel – The concrete optimization model instance.

Returns:

The original optimization model object.

Return type:

model