Advanced Features

Tutorials

Running simulations from file / Command line support

The mcmc::cmdint::CmdIntSim<SB, MS> provides an interface for setting up and executing simulations based on command line arguments to the executable.

The main function of the mcmc::cmdint::CmdIntSim struct:

void run(int argc, char **argv)
{
   if(argc > 1)
   {
      execute(argc, argv);
   }
   else
   {
      prepare();
   }
}

calls in the case of additional command line arguments the member function execute , which prepares or runs a MCMC simulation based on the provided template parameters systembase SB and measurement processor MS.

Command line arguments need to be passed in the following order:

./{executable} {execution_mode} {target_name} {sim_root_dir="./"} {rel_path=true} {run=true} {eval=true}

with the following variables:

• {executable} – name of the executable

• {execution_mode} – execution mode, one of equilibrium_time, correlation_time and expectation_value,

• {target_name} – identifier of the current simulation

• {sim_root_dir} – relative path to the project_root_dir (set by param_helper::proj::set_relative_path_to_project_root_dir("../")) or absolute path to the simulation directory used for storing the configuration, data and result files of the simulation

• {rel_path} – variable indicating whether sim_root_dir refers to an absolute or a relative path

• {run} – variable indicating whether the MCMC simulation is performed (true) or not (false)

• {eval} – variable indicated whether the evaluation should be performed in the same run (true) or not (false)

The virtual member function prepare can be overloaded to write all necessary configuration files for a MCMC simulation to file. In both cases, the PathParameters class is used in the background to set up the necessary directories in the filesystem.

The following code of a main.cpp file contains a minimal example for a simulation with command line support:

#include "../include/mcmcsystem/config.h"

#include <mcmc/mcmc_simulation/header.hpp>
#include <mcmc/mcmc_simulation/util/intervals.hpp>
#include <mcmc/modes/mode_header.hpp>
#include <mcmc/mcmc_simulation/util/random.hpp>

#include <command_line_support/cmdint.hpp>

#include "../include/mcmcsystem/mcmcsystem.hpp"


struct ExpectationValueSimulation : mcmc::cmdint::CmdIntSim<MCMCSystem, mcmc::measures::ReadableMeasure>
{
   using mcmc::cmdint::CmdIntSim<MCMCSystem, mcmc::measures::ReadableMeasure>::CmdIntSim;

   void prepare() override
   {
      auto sigma_intervals = mcmc::util::linspace(0.2, 1.0, 9);

      MCMCSystem system({1.0, 2.0, -1.5}, 1.0, 0.002);
      mcmc::measures::ReadableMeasure readable_measures(this->path_parameters_.get_rel_data_dir());

      auto simulation_parameters = mcmc::simulation::Simulation<MCMCSystem>::prepare_simulation_from_file(
            system, readable_measures,
            "systembase", "sigma", sigma_intervals);

      typedef mcmc::mode::ExpectationValue ExpectationValueParams;
      ExpectationValueParams expectation_value_parameters(
            10, 100000, 100, {"Config", "Mean"}, {}, "hot", "statistical");

      // Store simulation parameters
      simulation_parameters.write_to_file(this->path_parameters_.get_rel_config_dir());
      expectation_value_parameters.write_to_file(this->path_parameters_.get_rel_config_dir());
   }
};


int main(int argc, char **argv) {
   // Initialize project dependent parameters
   param_helper::proj::set_relative_path_to_project_root_dir("../");

#ifdef PYTHON_BACKEND
   mcmc::util::initialize_python(PYTHON_SCRIPTS_PATH);
#endif

   // Setting up and calling the main function for command line support
   ExpectationValueSimulation expectation_value_simulation("MCMCSystemSimulation", "./", true);
   expectation_value_simulation.run(argc, argv);

   // Finalization
#ifdef PYTHON_BACKEND
   mcmc::util::finalize_python();
#endif
   return 0;
}

Based on this setting, it is possible to, first, prepare a simulation by calling:

./{executable}

and to execute the simulation based on the command described above, afterwards.

The Ising model simulation in the examples/ directory, as well as template projects generated with the generate_application.py script and with command_line_support for the -mt parameter, represent additional examples for simulations with command line support.

Note that for the preparation, the static constructor mcmc::simulation::Simulation<SB>::prepare_simulation_from_file is used in combination with the particularly for this use case defined mcmc::mode::FromFilePreparation execution mode, corresponding to the default EP template argument of the mcmc::simulation::Simulation class.

Furthermore, note that if the prepare mode is not overloaded, running the executable with command line arguments generates configuration files based on the default constructors of the given MCMC system, measurement processor and evaluation mode. In a next step, the configuration files can be modified. Finally, by calling the same command again, the MCMC simulation will be executed.

Running simulations on a CPU / GPU cluster

The mcmc::cluster::execute function provides the possibility to prepare and to submit jobs for executing simulations on a CPU or GPU cluster.

For this to work, it is important to adapt for running simulation on a cpu cluster the mcmc::cluster::prepare_execution_on_cpu_cluster and mcmc::cluster::run_execution_on_cpu_cluster functions. Similar functions exist for executions on a GPU cluster.

In the existing code, we make use of two additional global variables, namely g_executable_name and g_cluster_mode. The two variables can be initialized by

mcmc::cluster::initialize_cluster_params(PROJECT_NAME, CLUSTER_MODE);

where PROJECT_NAME, CLUSTER_MODE are defined in the provided examples in a config.h file in the include directory which is generated by cmake.

By initializing the cluster parameters and by adding a function call of mcmc::cluster::execute to the example given above, the MCMC simulation can be executed by simple running:

./{executable}

where the script takes care of the following tasks:

• the prepare function generates the configuration files for the MCMC simulation and calls mcmc::cluster::execute with running_mode=RunningMode::prep_and_exec

• the mcmc::cluster::execute generates the required bash script for a submission to the cluster and submits the job to cluster afterwards

• when the job is executed on the cluster, the executable is called with command line arguments and the execute function of the command line support struct is called which executes the actual MCMC simulation.

The modified main.cpp file looks as follows:

#include "../include/mcmcsystem/config.h"

#include <mcmc/mcmc_simulation/header.hpp>
#include <mcmc/mcmc_simulation/util/intervals.hpp>
#include <mcmc/modes/mode_header.hpp>
#include <mcmc/mcmc_simulation/util/random.hpp>

#include <command_line_support/cmdint.hpp>
#include <cluster_support/cluster_integration.hpp>

#include "../include/mcmcsystem/mcmcsystem.hpp"


struct ExpectationValueSimulation : mcmc::cmdint::CmdIntSim<MCMCSystem, mcmc::measures::ReadableMeasure>
{
   using mcmc::cmdint::CmdIntSim<MCMCSystem, mcmc::measures::ReadableMeasure>::CmdIntSim;

   void prepare() override
   {
      auto sigma_intervals = mcmc::util::linspace(0.2, 1.0, 9);

      MCMCSystem system({1.0, 2.0, -1.5}, 1.0, 0.002);
      mcmc::measures::ReadableMeasure readable_measures(this->path_parameters_.get_rel_data_dir());

      auto simulation_parameters = mcmc::simulation::Simulation<MCMCSystem>::prepare_simulation_from_file(
            system, readable_measures,
            "systembase", "sigma", sigma_intervals);

      typedef mcmc::mode::ExpectationValue ExpectationValueParams;
      ExpectationValueParams expectation_value_parameters(
            10, 100000, 100, {"Config", "Mean"}, {}, "hot", "statistical");

      // Store simulation parameters
      simulation_parameters.write_to_file(this->path_parameters_.get_rel_config_dir());
      expectation_value_parameters.write_to_file(this->path_parameters_.get_rel_config_dir());

      // Prepare expectation value simulation on a cluster and submit the job with one function call
      mcmc::cluster::execute<MCMCSystem, mcmc::measures::ReadableMeasure>(
            "expectation_value", this->path_parameters_, true, true,
            mcmc::cluster::Device::on_cpu_cluster, mcmc::cluster::RunningMode::RunningMode::prep_and_exec, {});
   }
};


int main(int argc, char **argv) {
   // Initialize project dependent parameters
   param_helper::proj::set_relative_path_to_project_root_dir("../");

#ifdef PYTHON_BACKEND
   mcmc::util::initialize_python(PYTHON_SCRIPTS_PATH);
#endif
   mcmc::cluster::initialize_cluster_params(PROJECT_NAME, CLUSTER_MODE);

   // Setting up and calling the main function for command line support
   ExpectationValueSimulation expectation_value_simulation("MCMCSystemSimulation", "./", true);
   expectation_value_simulation.run(argc, argv);

   // Finalization
#ifdef PYTHON_BACKEND
   mcmc::util::finalize_python();
#endif
   return 0;
}

Note that a template project with a main.cpp script with cluster support can also be generated by the generate_application.py script with cluster_support for the -mt parameter.

Command line support

FromFilePreparation

class FromFilePreparation : public Parameters

Alternative execution mode which can be used in combination with the static constructor mcmc::simulation::Simulation<SB>::prepare_simulation_from_file to generate .json configuration files for preparing later runs with actual execution modes. Note that running a MCMC simulation with this execution mode is meaningless.

Path parameters

struct mcmc::cmdint::PathParameters

Helper class for retrieving all important directories needed for simulations based on arguments in the terminal or for simulations on a cluster.

Public Functions

inline PathParameters(const std::string &target_name, const std::string &sim_root_dir = "./", const bool rel_path = true)

Constructor providing the necessary information for a correct storage of configuration, data and result files.

Parameters

• target_name – Identifier of the current simulation (the directory is used for storing files in sim_root_dir/config/, sim_root_dir/data/, sim_root_dir/results/, sim_root_dir/cpu_cluster_runs/, sim_root_dir/gpu_cluster_runs/ throughout the simulation)

• sim_root_dir – Relative path to the project_root_dir (set by param_helper::proj::set_relative_path_to_project_root_dir("../")) or absolute path to the simulation directory used for storing the configuration, data and result files of the simulation

• rel_path – Variable indicating whether sim_root_dir refers to an absolute (false) or a relative (true) path

inline std::string get_rel_config_dir() const

Returns the rel_config_dir and creates the directory if not present.

inline std::string get_rel_data_dir() const

Returns the rel_data_dir and creates the directory if not present.

inline std::string get_rel_results_dir() const

Returns the rel_results_dir and creates the directory if not present.

inline std::string get_rel_cpu_bash_script_dir() const

Returns the rel_cpu_bash_script_dir and creates the directory if not present.

inline std::string get_rel_gpu_bash_script_dir() const

Returns the rel_gpu_bash_script_dir and creates the directory if not present.

Command line interface

template<typename ExecutionParams, typename SB, typename MS>
void mcmc::cmdint::prep_default_execution(const mcmc::cmdint::PathParameters &path_parameters)

Helper function for preparing a simulation based on the default arguments of the respective template class arguments.

Parameters

path_parameters – Path parameters containing all important paths for finding the necessary configurations files and for storing data and results.

template<typename SB, typename MS>
void mcmc::cmdint::execute(const std::string &mode_type, const mcmc::cmdint::PathParameters &path_parameters, const bool run = true, const bool eval = true)

Helper function for preparing and executing simulations based on a mode type and path parameters.

Parameters

• mode_type – The execution mode used for the simulation ("equilibrium_time", "correlation_time" or "expectation_value")

• path_paramers – Path parameters containing all important paths for finding the necessary configuration files and for storing data and results

• run – Indicate whether the MCMC simulation is performed (true) or not (false)

• eval – Indicate whether the evaluation should be performed in the same run (true) or not (false)

template<typename SB, typename MS>
struct mcmc::cmdint::CmdIntSim

Base strcut for preparing and executing a simulation from the command line.

Public Functions

inline CmdIntSim(const std::string &target_name, const std::string &sim_root_dir = "./", const bool rel_path = true)

Constructor for providing all important path variables.

Parameters

• target_name – Identifier of the current simulation (the directory is used for storing files in sim_root_dir/config/, sim_root_dir/data/, sim_root_dir/results/, sim_root_dir/cpu_cluster_runs/, sim_root_dir/gpu_cluster_runs/ throughout the simulation)

• sim_root_dir – Relative path to the project_root_dir (set by param_helper::proj::set_relative_path_to_project_root_dir("../")) or absolute path to the simulation directory used for storing the configuration, data and result files of the simulation

• rel_path – Variable indicating whether sim_root_dir refers to an absolute (false) or a relative (true) path

inline virtual void prepare()

Virtual method which is supposed to be overloaded.

Helpful for a preparation of the simulation or an immediate execution (on cpu/gpu/Device::locally, testing/running directly)

Cluster support

Cluster parameter initialization

void mcmc::cluster::initialize_cluster_params(const std::string &executable_name, const std::string &cluster_mode)

Function for intializing project dependent parameters for an execution of the MCMC simulation on a cluster.

If you generate your project with the generate_application.py script, the CLUSTER_MODE CMake variable can be set when calling cmake. In this setting, the recommended way to initialize the cluster parameters is to add the following line at the beginning of your main function.

mcmc::cluster::initialize_cluster_params(PROJECT_NAME, CLUSTER_MODE);

The two variables, PROJECT_NAME and the CLUSTER_MODE are defined in the config.h file in the include/ which is automatically generated by cmake.

Parameters

• executable_name – Name of the executable

• cluster_mode – Variable allowing to test the submission to the cluster Device::locally before an actual submission to the cluster. By setting cluster_mode="local", the job will be started by executing

  nice -n 17 bash <name_of_the_job_file>.sh.
  

  With cluster_mode="on_cluster", the job will be submitted to the cluster (for more details, see also mcmc::cluster::run_execution_on_cpu_cluster and mcmc::cluster::run_execution_on_gpu_cluster).

Simulations on a cluster

template<typename SBP, typename MS>
void mcmc::cluster::execute(const std::string &mode_type, const mcmc::cmdint::PathParameters &path_parameters, const bool run = true, const bool eval = true, const Device device = Device::locally, const RunningMode running_mode = RunningMode::prep_and_exec, const std::vector<std::string> &additional_args = {})

Main function for preparing and executing MCMC simulations on a cluster.

Note that the function supports with the help of device=Device::locally the possibility to execute exactly the same code as the one being executed on the cluster beforehand Device::locally.

Parameters

• mode_type – The execution mode used for the simulation ("equilibrium_time", "correlation_time" or "expectation_value")

• path_parameters – Path parameters containing all important paths for finding the necessary configuration files and for storing data and results

• run – Indicate whether the MCMC simulation is performed (true) or not (false)

• eval – Indicate whether the evaluation should be performed in the same run (true) or not (false)

• device – Indicate the device to be used (Device::locally, Device::on_gpu_cluster or Device::on_cpu_cluster) by choosing Device::locally, it can be checked if the simulation runs properly before an actual simulation on the cluster. This is in particular useful to avoid unnecessary submissions to the cluster.

• running_mode – Indicate whether a submission on the cluster should be:

  • prepared (RunningMode::prep), referring to a generation of the .sh file for execution,

  • executed (RunningMode::exec), referring to a submission of the job to the cluster,

  • or prepared and executed (RunningMode::prep_and_exec), referring to a generation of the .sh file and a subsequent submission to the cluster.

• addition_args – List of optional additional args to be passed to the C++ executable when the simulation is executed.

CPU cluster integration

void mcmc::cluster::prepare_execution_on_cpu_cluster(const std::string &mode_type, const mcmc::cmdint::PathParameters &path_parameters, const std::string &executable_name, const bool run = true, const bool eval = true, const std::vector<std::string> &additional_args = {})

Function used by mcmc::cluster::execute to prepare the bash script for executing a simulation on a cpu cluster. Note that the function needs to be adapted according to the used cpu cluster.

void mcmc::cluster::run_execution_on_cpu_cluster(const std::string &mode_type, const mcmc::cmdint::PathParameters &path_parameters, std::string_view cluster_mode)

Function used by mcmc::cluster::execute to submit the job to a cpu cluster. Note that the function needs to be adapted according to the used cpu cluster.

GPU cluster integration

void mcmc::cluster::prepare_execution_on_gpu_cluster(const std::string &mode_type, const mcmc::cmdint::PathParameters &path_parameters, const std::string &executable_name, const bool run = true, const bool eval = true, const std::vector<std::string> &additional_args = {})

Function used by mcmc::cluster::execute to prepare the bash script for executing a simulation on a gpu cluster. Note that the function needs to be adapted according to the used gpu cluster.

void mcmc::cluster::run_execution_on_gpu_cluster(const std::string &mode_type, const mcmc::cmdint::PathParameters &path_parameters, std::string_view cluster_mode)

Function used by mcmc::cluster::execute to submit the job to a gpu cluster. Note that the function needs to be adapted according to the used gpu cluster.

Virtual environment integration

The MCMCSimulationLib provides the possibility to pass the conda activate path and the name of the used virtual environment to C++, since knowing these variables can be necessary for generating the bash scripts for a submission to a cluster:

static std::string mcmc::virtualenv::virtual_env_integration::g_conda_activate_path = CONDA_ACTIVATE_PATH

Path to conda activate (for example: ~/miniconda3/bin/activate). Per default, this path is set to the CMake variable CONDA_ACTIVATE_PATH.

static std::string mcmc::virtualenv::virtual_env_integration::g_virtual_env = VIRTUAL_ENV

Name of the virtual environment to be activated before execution of the script. Per default, this path is set to the CMake variable VIRTUAL_ENV.

As explained in the Installation section, the variables can be set globally by providing respective CMake variables. Alternatively, the following functions can be used to temporarily change these variables:

static void mcmc::virtualenv::virtual_env_integration::set_conda_activate_path(std::string_view conda_activate_path)

Can be used to set the conda_activate_path to a different than the default one.

Parameters

conda_activate_path – Path to conda activate

static void mcmc::virtualenv::virtual_env_integration::set_virtual_env(std::string_view virtual_env)

Can be used to set the virtual environment to a different than the default one.

Parameters

virtual_env – Name of the virtual environment to be activated