ETM

ETM is a Dutch-based national energy system model focusing on energy transition. Refer to https://multimodelling.readthedocs.io/en/latest/energy_models/ETM/index.html for more overall information regarding the model.

General model information

General model information questions were asked regarding basic information, model versions, and point of contact for questions. The ETM model is developed and maintained by Quintel.

Questions to ask	Answers/Explanation
Model name	ETM (Energy Transition Model)
Model owner	Quintel
Model Developer	Quintel
The latest model version/date
The model version used in this project	Latest version
Organization	Quintel
Individual	Chael Kruip

A second set of questions was asked regarding whether the model is type or token, the intended purpose of the model, and the level of decision that the model aims to support. We understand that the model can be categorized as type and token model. The model’s intended purpose is to analyze the energy system of the Netherlands. Long-term national policy targets are emission reductions, efficiency, and renewable energy production. The medium-term focus is on flexibility analysis and annual demand requirements under different ‘if’ conditions.

Questions to ask	Answers/Explanation
Is the model a token model? If so, give illustration(s).	Users/modelers can define specific parts of the energy system in detail
Is the model a type model? If so, give illustration(s).	It is an energy domain-specific model that can be configured to represent a specific target system (ETM is a framework: model structure predefined; can be initialized for different energy systems; can be applied to sub-systems)
Briefly describe the intended purpose of the model	Energy System Analysis of the Netherlands
Strategic - long-term planning; what do we want?	1. Long-term national policy targets related to emissions reductions, efficiency, renewable energy production 2. Long-term targets of, for example, production for industries, including subsectors, sectoral demands, energy infrastructure capacity
Tactical - medium-term; how do we approach this?	Flexibility analysis 2. Annual demand requirement under different ‘if’ conditions
Operational - short-term; regular/ day-to-day operations?

Typical questions about the model include future flexibility in a national energy system or electrification volume needed to replace heat demand in the built environment. The model is handy for quickly exploring and quantifying potential future energy systems in detail. The model is free to use, open-source, and applicable to national and regional contexts within the EU. One of the major limitations of the model is the lack of complex interactions between different components of an integrated energy system. The governments at different levels within the Netherlands have used the model for regional and national energy transition analysis. The model has been pushed to achieve a given political agenda, which is a case of the model not being used for its intended purpose.

Questions to ask	Answers/Explanation
What are typical types of questions that can be asked to the model? provide examples of such questions	What flexibility is needed? 2. What is the heat demand for electrifying the built environment? 3. How many petrol-fueled vehicles need to be replaced to reduce mobility emissions by x %? Etc.
What are the strengths of this model? What is unique?	1. A handy calculation tool allowing for quick exploration and quantification of potential future energy systems in detail. 2. The model can perform analysis at different geographical levels with ease. 3. The model covers various aspects of the energy system, such as demand, supply, and emissions. 4. The model is free to use, open-source, and available for EU countries, municipalities, and many other regions.
What are the important limitations of the model?	1. The model does not consider the complex interactions within the energy system 2. Social interaction and impacts missing
Cases/examples where the model was used for its intended purpose	1. The governments at different levels within the Netherlands have used the model for regional and national energy transition analysis. 2. The model has been applied/used in collaboration with industries and universities to understand sectoral energy demands, energy supplies from different technology options, and energy balances.
Cases/examples where the model was not used for its intended purpose; are there any examples of model abuse or misuse?	The model has been pushed to achieve a given political agenda/goal (the assumptions/input is an input, which is the user’s responsibility) and interpretation of the result.

The next set of questions is related to model documentation, accessibility, and type. The model documentation is not complete but adequate. The documentation is available online and is in English. The graphical user interface (GUI) and Application Programming Interface (API) are online. The model is static, deterministic, and continuous.

Questions to ask	Answers/Explanation
Is the model documentation complete?	The documentation is not complete but adequate.
Is the documentation accessible? If so, how?	Yes https://docs.energytransitionmodel.com/main/intro/
Is the documentation in English?	Yes
Does the model have a GUI? If so, how to access it?	Yes, the GUI is online. https://energytransitionmodel.com/
Does the model have an Application Programming Interface (API) ? If so, how to access it?	Yes, APIs are also online. https://docs.energytransitionmodel.com/api/intro
Is the model static or dynamic?	Static Additional comments/remarks: The model has a static start-to-end date calculation. Energy storage and market principles are dynamic time steps.
Is the model continuous or discrete?	continuous
Is the model stochastic or deterministic?	Deterministic
Is it an optimization model? If so, what type of algorithms it uses?	No

The next set of questions are regarding the modeling paradigm, implementation environment, and license. The model applies multiple formalisms, such as mathematical equations, object-oriented programming, etc. Multiple general-purpose programming languages, such as Python, JAVA, SQL, etc., are used. The model is implemented in an Excel spreadsheet and does not require any license to run.

Questions to ask	Answers/Explanation
What modeling paradigm or formalism does the model use?	Mathematical equations (translation of UI input to model input; graph query), procedural (mostly) and functional (some), Object-oriented, etc.
Is it implemented in a General purpose programming language?	Python, JAVA, Ruby (mostly), SQL database, and C++ for optimized/memory-intensive activity
Does it use a modeling/Simulation environment/package?	No
Is it implemented in a spreadsheet?	Excel
Is any license required to run the model?	No

Model content

A preliminary set of model content questions were related to energy system integration and scope. The model represents an integrated energy system.

Essential elements and concepts included in the model are energy-demanding sectors, energy-supplying options, energy infrastructure, and fuel feedstock. The model covers a wide range of flexibility options, for example, technologies accommodating large fluctuations in volume such as power-to-gas (P2G) or gas storage and large sudden fluctuations in capacities such as heat and power plants.

Questions to ask	Answers/Explanation
Does the model represent an integrated energy system?	Yes
What important elements and concepts are included in the model?	1. Covers the entire energy system of the Netherlands 2. Content-wise coverage: Energy-demanding sectors (built environment, industries, agriculture, and mobility), energy supply options (for example, wind, solar, biomass, geothermal, and non-renewable sources), energy infrastructure (electricity, heat, gas, hydrogen, and CO2), and fuel feedstock
What elements and concepts are currently not included in the model, but in your opinion, those shall be included?
Specific attention to flexibility options: What type of flexibility options are included in the model?	A wide range of flexibility options are included: a. large fluctuations in volume (P2G, Import/export or storage of gas/hydrogen, and seasonal storage of heat) b. large or sudden fluctuations in capacity (storage in batteries, dispatchable heat and power plants, and demand side response) c. Volume and capacity fluctuations (import/export of electricity, P2H, curtailment of renewable electricity production, and large-scale electricity storage)

The next set of content-related questions included scale and resolution. The spatial scale of the model is the national level. The model has a long-term temporal scale till 2050; however, the emphasis is till 2050. The spatial resolution is at the city or municipality level. Temporal resolution is an hour.

Questions to ask	Answers/Explanation
What spatial (or geospatial) scale does the model have?	National
What temporal (or time) scale does the model have?	Long-term (till 2070) However, the emphasis is till 2050.
Spatial resolution	Municipality
Temporal resolution	hourly

The next set of questions is related to model assumptions, model inputs, parameters, and outputs, and data sources related to the model. Most energy balances happen annually, allowing the model to provide quick results for different scenarios. The model does not differentiate between different temperature levels, which others might contest as industries require high-temperature heat, and the built environment uses low-temperature heat. The input is through sliders at the GUI, and the output results are graphs visualized through the GUI. Some important model inputs are sectoral energy and services demand, supply options, and profiles. Important model outputs are final energy demands and supplies, investments in technology options, yearly cost of energy production, etc. Links to some of the data sources have been provided. Data can be shared, and some links for that are provided.

Questions to ask	Answers/Explanation
What critical assumptions does the model have?	1. Most energy balances happen annually, which allows the model to provide quick results for different scenarios 2. Multiple versions of the II3050 scenario are considered in the model.
Which ones are likely to be contested by others? Why?	1. No differentiation between temperature levels; only one type of heat, which is not realistic. Industry uses high-temperature heat, and the built environment uses low-temperature heat 2. In dispatchable power plants, there is no ramping speed
What is/are the model input format(s)?	Input is through sliders at the GUI.
What is/are the model output format(s)?	Output results are graphs visualized at the GUI.
What are the important model inputs?	674 input variables Examples: sectoral energy and services demand (households, buildings, transportation, industry, agriculture, etc.), supply (electricity, district heating, hydrogen, transport fuels, etc.), profiles (demand, supply, prices, etc.), etc.
What important parameters does the model have?	Technology- and process-related parameters (for example, efficiency), emission factors, etc.
What are the important model outputs?	Final energy demands and supply, investment in technology options, hourly electricity prices, yearly energy system cost, production, etc.
What are the data sources used by the model?	Some links to data sources: https://data.energytransitionmodel.com/ https://github.com/quintel/etsource
Any data that can be shared? If so, what and how to access them?	Yes https://refman.energytransitionmodel.com/ https://github.com/quintel/etdataset-public

Continuing with the model content, there were questions regarding verification, validation, and test, and uncertainty descriptions. The model works in a test-driven development environment. Unit testing is done for low-level functions. Model inputs, model structure, and data consistency are verified, tested, and validated. The effect of policies on the inputs is tested. The qualitative method of validating is expert consultation. One of the quantitative methods deployed by the model is a comparison with other models and pilot runs. No systematic uncertainty verification methods exist, though sensitivity analyses are performed on various input parameters.

Questions to ask	Answers/Explanation
Can you comment on the test coverage of the model?	Test-driven development, unit testing for low-level functions, integration test
What is being verified, validated, or tested in the model?	1. Input, model structure, data consistency, etc.. 2. The possible effect of policies is given as input to the model
What methods are used for the model verification, validation, and testing, if any?	1. Qualitative method: expert validation 2. Quantitative method: comparison with other models with more significant details, pilot runs Simulate a user of the model Incremental, iterative Etc.
Can you comment on the uncertainty in model parameters?	Sensitivity analyses; no systematic uncertainty verification method
Can you comment on the uncertainty in model input?
Can you comment on the uncertainty in the model structure?