pymake package

Subpackages

Submodules

Module contents

pymake is a python package for compiling MODFLOW-based and other Fortran, C, and C++ programs. The package determines the build order using a directed acyclic graph and then compiles the source files using GNU compilers (gcc, g++, gfortran) or Intel compilers (ifort, icc).

class Pymake(name='pymake', verbose=None)[source]

Bases: object

Pymake class for interacting with pymake functionality. This is essentially a wrapper for all of the pymake functions needed to download and build a target.

argv_reset_settings(args)[source]

Reset settings using command line arguments

Parameters

args (Namespace object) – reset self.variables using command line arguments

build(target=None, srcdir=None, modify_exe_name=False)[source]

Build the target

Parameters
  • target (str) – target name. If target is None self.target is used. (default is None)

  • srcdir (str) – path to directory with source files. (default is None)

  • modify_exe_name (bool) – boolean that determines if the target name can be modified to include precision (dbl) and debugging (d) indicators.

compress_targets()[source]

Compress targets in build_targets list.

download_setup(target, url=None, download_path='.', verify=True, timeout=30)[source]

Setup download

Parameters
  • target (str) – target name

  • url (str) – url of asset

  • download_path (str) – path where the asset will be saved

  • verify (bool) – boolean defining ssl verification

  • timeout (int) – download timeout in seconds (default is 30)

download_target(target, url=None, download_path='.', verify=True, timeout=30)[source]

Setup and download url

Parameters
  • target (str) – target name

  • url (str) – url of asset

  • download_path (str) – path where the asset will be saved

  • verify (bool) – boolean defining ssl verification

  • timeout (int) – download timeout in seconds (default is 30)

Returns

success – boolean flag indicating download success

Return type

bool

download_url()[source]

Download files from the url

Returns

success – boolean flag indicating download success

Return type

bool

finalize()[source]

Finalize Pymake class

reset(target)[source]

Reset PyMake object variables for a target

Parameters

target (str) – target name

set_build_target_bool(target=None)[source]

Evaluate if the executable exists and if so and the command line argument –keep is specified then the executable is not built.

Parameters

target (str) – target name. If target is None self.target will be used. (default is None)

Returns

build – boolean indicating if the executable should be built

Return type

bool

update_build_targets()[source]

Add target to build_targets list if it is not in the list

update_target(target, modify_target=False)[source]

Update target name with executable extension on Windows and based on pymake settings.

Parameters
  • target (str) – target name

  • modify_target (bool) – boolean indicating if the target name can be modified based on pymake double and debug settings (default is False)

Returns

target – updated target name

Return type

str

build_apps(targets=None, pymake_object=None, download_dir=None, appdir=None, verbose=None, release_precision=True, meson=False, mesondir='.', clean=True)[source]

Build all of the current targets or a subset of targets.

Parameters
  • targets (str or list of str) – targets to build. If targets is None, all current targets will be build. Default is None

  • pymake_object (Pymake()) – Pymake object created outside of build_apps

  • download_dir (str) – download directory path

  • appdir (str) – target path

  • release_precision (bool) – boolean indicating if only the release precision version should be build. If release_precision is False, then the release precision version will be compiled along with a double precision version of the program for programs where the standard_switch and double_switch in usgsprograms.txt is True. default is True.

  • meson (bool) – boolean indicating that the executable should be built using the meson build system. (default is False)

  • mesondir (str) – Main meson.build file path

  • clean (bool) – boolean determining of final download should be removed

Returns

returncode – integer value indicating successful completion (0) or failure (>0)

Return type

int

compare(namefile1, namefile2, precision='auto', max_cumpd=0.01, max_incpd=0.01, htol=0.001, outfile1=None, outfile2=None, files1=None, files2=None)[source]

Compare the budget and head results for two MODFLOW-based model simulations.

Parameters
  • namefile1 (str) – namefile path for base model

  • namefile2 (str) – namefile path for comparison model

  • precision (str) – precision for binary head file (“auto”, “single”, or “double”) default is “auto”

  • max_cumpd (float) – maximum percent discrepancy allowed for cumulative budget terms (default is 0.01)

  • max_incpd (float) – maximum percent discrepancy allowed for incremental budget terms (default is 0.01)

  • htol (float) – maximum allowed head difference (default is 0.001)

  • outfile1 (str) – budget comparison output file name. If outfile1 is None, no budget comparison output is saved. (default is None)

  • outfile2 (str) – head comparison output file name. If outfile2 is None, no head comparison output is saved. (default is None)

  • files1 (str) – base model output file. If files1 is not None, results will be extracted from files1 and namefile1 will not be used. (default is None)

  • files2 (str) – comparison model output file. If files2 is not None, results will be extracted from files2 and namefile2 will not be used. (default is None)

Returns

success – boolean indicating if the budget and head differences are less than max_cumpd, max_incpd, and htol.

Return type

bool

compare_budget(namefile1, namefile2, max_cumpd=0.01, max_incpd=0.01, outfile=None, files1=None, files2=None)[source]

Compare the budget results from two simulations.

Parameters
  • namefile1 (str) – namefile path for base model

  • namefile2 (str) – namefile path for comparison model

  • max_cumpd (float) – maximum percent discrepancy allowed for cumulative budget terms (default is 0.01)

  • max_incpd (float) – maximum percent discrepancy allowed for incremental budget terms (default is 0.01)

  • outfile (str) – budget comparison output file name. If outfile is None, no comparison output is saved. (default is None)

  • files1 (str) – base model output file. If files1 is not None, results will be extracted from files1 and namefile1 will not be used. (default is None)

  • files2 (str) – comparison model output file. If files2 is not None, results will be extracted from files2 and namefile2 will not be used. (default is None)

Returns

success – boolean indicating if the difference between budgets are less than max_cumpd and max_incpd

Return type

bool

compare_concs(namefile1, namefile2, precision='auto', ctol=0.001, outfile=None, files1=None, files2=None, difftol=False, verbose=False)[source]

Compare the mt3dms and mt3dusgs concentration results from two simulations.

Parameters
  • namefile1 (str) – namefile path for base model

  • namefile2 (str) – namefile path for comparison model

  • precision (str) – precision for binary head file (“auto”, “single”, or “double”) default is “auto”

  • ctol (float) – maximum allowed concentration difference (default is 0.001)

  • outfile (str) – concentration comparison output file name. If outfile is None, no comparison output is saved. (default is None)

  • files1 (str) – base model output file. If files1 is not None, results will be extracted from files1 and namefile1 will not be used. (default is None)

  • files2 (str) – comparison model output file. If files2 is not None, results will be extracted from files2 and namefile2 will not be used. (default is None)

  • difftol (bool) – boolean determining if the absolute value of the concentration difference greater than ctol should be evaluated (default is False)

  • verbose (bool) – boolean indicating if verbose output should be written to the terminal (default is False)

Returns

success – boolean indicating if the concentration differences are less than ctol.

Return type

bool

compare_heads(namefile1, namefile2, precision='auto', text='head', text2=None, htol=0.001, outfile=None, files1=None, files2=None, difftol=False, verbose=False, exfile=None, exarr=None, maxerr=None)[source]

Compare the head results from two simulations.

Parameters
  • namefile1 (str) – namefile path for base model

  • namefile2 (str) – namefile path for comparison model

  • precision (str) – precision for binary head file (“auto”, “single”, or “double”) default is “auto”

  • htol (float) – maximum allowed head difference (default is 0.001)

  • outfile (str) – head comparison output file name. If outfile is None, no comparison output is saved. (default is None)

  • files1 (str) – base model output file. If files1 is not None, results will be extracted from files1 and namefile1 will not be used. (default is None)

  • files2 (str) – comparison model output file. If files2 is not None, results will be extracted from files2 and namefile2 will not be used. (default is None)

  • difftol (bool) – boolean determining if the absolute value of the head difference greater than htol should be evaluated (default is False)

  • verbose (bool) – boolean indicating if verbose output should be written to the terminal (default is False)

  • exfile (str) – path to a file with exclusion array data. Head differences will not be evaluated where exclusion array values are greater than zero. (default is None)

  • exarr (numpy.ndarry) – exclusion array. Head differences will not be evaluated where exclusion array values are greater than zero. (default is None).

  • maxerr (int) – maximum number of head difference greater than htol that should be reported. If maxerr is None, all head difference greater than htol will be reported. (default is None)

Returns

success – boolean indicating if the head differences are less than htol.

Return type

bool

compare_stages(namefile1=None, namefile2=None, files1=None, files2=None, htol=0.001, outfile=None, difftol=False, verbose=False)[source]

Compare SWR process stage results from two simulations.

Parameters
  • namefile1 (str) – namefile path for base model

  • namefile2 (str) – namefile path for comparison model

  • precision (str) – precision for binary head file (“auto”, “single”, or “double”) default is “auto”

  • htol (float) – maximum allowed stage difference (default is 0.001)

  • outfile (str) – head comparison output file name. If outfile is None, no comparison output is saved. (default is None)

  • files1 (str) – base model output file. If files1 is not None, results will be extracted from files1 and namefile1 will not be used. (default is None)

  • files2 (str) – comparison model output file. If files2 is not None, results will be extracted from files2 and namefile2 will not be used. (default is None)

  • difftol (bool) – boolean determining if the absolute value of the stage difference greater than htol should be evaluated (default is False)

  • verbose (bool) – boolean indicating if verbose output should be written to the terminal (default is False)

Returns

success – boolean indicating if the stage differences are less than htol.

Return type

bool

compare_swrbudget(namefile1, namefile2, max_cumpd=0.01, max_incpd=0.01, outfile=None, files1=None, files2=None)[source]

Compare the SWR budget results from two simulations.

Parameters
  • namefile1 (str) – namefile path for base model

  • namefile2 (str) – namefile path for comparison model

  • max_cumpd (float) – maximum percent discrepancy allowed for cumulative budget terms (default is 0.01)

  • max_incpd (float) – maximum percent discrepancy allowed for incremental budget terms (default is 0.01)

  • outfile (str) – budget comparison output file name. If outfile is None, no comparison output is saved. (default is None)

  • files1 (str) – base model output file. If files1 is not None, results will be extracted from files1 and namefile1 will not be used. (default is None)

  • files2 (str) – comparison model output file. If files2 is not None, results will be extracted from files2 and namefile2 will not be used. (default is None)

Returns

success – boolean indicating if the difference between budgets are less than max_cumpd and max_incpd

Return type

bool

download_and_unzip(url, pth='./', delete_zip=True, verify=True, timeout=30, max_requests=10, chunk_size=2048000, verbose=False)[source]

Download and unzip a zip file from a url.

Parameters
  • url (str) – url address for the zip file

  • pth (str) – path where the zip file will be saved (default is the current path)

  • delete_zip (bool) – boolean indicating if the zip file should be deleted after it is unzipped (default is True)

  • verify (bool) – boolean indicating if the url request should be verified

  • timeout (int) – url request time out length (default is 30 seconds)

  • max_requests (int) – number of url download request attempts (default is 10)

  • chunk_size (int) – maximum url download request chunk size (default is 2048000 bytes)

  • verbose (bool) – boolean indicating if output will be printed to the terminal

get_entries_from_namefile(namefile, ftype=None, unit=None, extension=None)[source]

Get entries from a namefile. Can select using FTYPE, UNIT, or file extension.

Parameters
  • namefile (str) – path to a MODFLOW-based model name file

  • ftype (str) – package type

  • unit (int) – file unit number

  • extension (str) – file extension

Returns

entries – list of tuples containing FTYPE, UNIT, FNAME, STATUS for each namefile entry that meets a user-specified value.

Return type

list of tuples

get_input_files(namefile)[source]

Return a list of all the input files in this model.

Parameters

namefile (str) – path to a MODFLOW-based model name file

Returns

filelist – list of MODFLOW-based model input files

Return type

list

get_mf6_blockdata(f, blockstr)[source]

Return list with all non comments between start and end of block specified by blockstr.

Parameters
  • f (file object) – open file object

  • blockstr (str) – name of block to search

Returns

data – list of data in specified block

Return type

list

get_mf6_comparison(src)[source]

Determine comparison type for MODFLOW 6 simulation.

Parameters

src (str) – directory path to search for comparison types

Returns

action – comparison type

Return type

str

get_mf6_files(mfnamefile)[source]

Return a list of all the MODFLOW 6 input and output files in this model.

Parameters

mfnamefile (str) – path to the MODFLOW 6 simulation name file

Returns

  • filelist (list) – list of MODFLOW 6 input files in a simulation

  • outplist (list) – list of MODFLOW 6 output files in a simulation

get_mf6_ftypes(namefile, ftypekeys)[source]

Return a list of FTYPES that are in the name file and in ftypekeys.

Parameters
  • namefile (str) – path to a MODFLOW 6 name file

  • ftypekeys (list) – list of desired FTYPEs

Returns

ftypes – list of FTYPES that match ftypekeys in namefile

Return type

list

get_mf6_mshape(disfile)[source]

Return the shape of the MODFLOW 6 model.

Parameters

disfile (str) – path to a MODFLOW 6 discretization file

Returns

mshape – tuple with the shape of the MODFLOW 6 model.

Return type

tuple

get_mf6_nper(tdisfile)[source]

Return the number of stress periods in the MODFLOW 6 model.

Parameters

tdisfile (str) – path to the TDIS file

Returns

nper – number of stress periods in the simulation

Return type

int

get_namefiles(pth, exclude=None)[source]

Search through a path (pth) for all .nam files.

Parameters
  • pth (str) – path to model files

  • exclude (str or lst) – File or list of files to exclude from the search (default is None)

Returns

namefiles – List of namefiles with paths

Return type

lst

get_repo_assets(github_repo=None, version=None, error_return=False, verify=True)[source]

Return a dictionary containing the file name and the link to the asset contained in a github repository.

Parameters
  • github_repo (str) – Repository name, such as MODFLOW-USGS/modflow6. If github_repo is None set to ‘MODFLOW-USGS/executables’

  • version (str) – github repository release tag

  • error_return (bool) – boolean indicating if None will be returned if there are GitHub API issues

  • verify (bool) – boolean indicating if the url request should be verified

Returns

result_dict – dictionary of file names and links

Return type

dict

get_sim_name(namefiles, rootpth=None)[source]

Get simulation name.

Parameters
  • namefiles (str or list of strings) – path(s) to MODFLOW-based model name files

  • rootpth (str) – optional root directory path (default is None)

Returns

simname – list of namefiles without the file extension

Return type

list

getmfexes(pth='.', version=None, platform=None, exes=None, verbose=False, verify=True)[source]

Get the latest MODFLOW binary executables from a github site (https://github.com/MODFLOW-USGS/executables) for the specified operating system and put them in the specified path.

Parameters
  • pth (str) – Location to put the executables (default is current working directory)

  • version (str) – Version of the MODFLOW-USGS/executables release to use. If version is None the github repo will be queried for the version number.

  • platform (str) – Platform that will run the executables. Valid values include mac, linux, win32 and win64. If platform is None, then routine will download the latest asset from the github repository.

  • exes (str or list of strings) – executable or list of executables to retain

  • verbose (bool) – boolean indicating if output will be printed to the terminal

  • verify (bool) – boolean indicating if the url request should be verified

main(srcdir=None, target=None, fc='gfortran', cc='gcc', makeclean=True, expedite=False, dryrun=False, double=False, debug=False, include_subdirs=False, fflags=None, cflags=None, syslibs=None, arch='intel64', makefile=False, makefiledir='.', srcdir2=None, extrafiles=None, excludefiles=None, sharedobject=False, appdir=None, verbose=False, inplace=False, networkx=False, meson=False, mesondir='.')[source]

Main pymake function.

Parameters
  • srcdir (str) – path for directory containing source files

  • target (str) – executable name or path for executable to create

  • fc (str) – fortran compiler

  • cc (str) – c or cpp compiler

  • makeclean (bool) – boolean indicating if intermediate files should be cleaned up after successful build

  • expedite (bool) – boolean indicating if only out of date source files will be compiled. Clean must not have been used on previous build.

  • dryrun (bool) – boolean indicating if source files should be compiled. Files will be deleted, if makeclean is True.

  • double (bool) – boolean indicating a compiler switch will be used to create an executable with double precision real variables.

  • debug (bool) – boolean indicating is a debug executable will be built

  • include_subdirs (bool) – boolean indicating source files in srcdir subdirectories should be included in the build

  • fflags (list) – user provided list of fortran compiler flags

  • cflags (list) – user provided list of c or cpp compiler flags

  • syslibs (list) – user provided syslibs

  • arch (str) – Architecture to use for Intel Compilers on Windows (default is intel64)

  • makefile (bool) – boolean indicating if a GNU make makefile should be created

  • makefiledir (str) – GNU make makefile path

  • srcdir2 (str) – additional directory with common source files.

  • extrafiles (str) – path for extrafiles file that contains paths to additional source files to include

  • excludefiles (str) – path for excludefiles file that contains filename of source files to exclude from the build

  • sharedobject (bool) – boolean indicating a shared object will be built

  • appdir (str) – path for executable

  • verbose (bool) – boolean indicating if output will be printed to the terminal

  • inplace (bool) – boolean indicating that the source files in srcdir, srcdir2, and defined in extrafiles will be used directly. If inplace is False, source files will be copied to a directory named srcdir_temp. (default is False)

  • networkx (bool) – boolean indicating that the NetworkX python package will be used to create the Directed Acyclic Graph (DAG) used to determine the order source files are compiled in. The NetworkX package tends to result in a unique DAG more often than the standard algorithm used in pymake. (default is False)

  • meson (bool) – boolean indicating that the executable should be built using the meson build system. (default is False)

  • mesondir (str) – Main meson.build file path

Returns

returncode – return code

Return type

int

make_plots(srcdir, outdir, include_subdir=False, level=3, extension='.png', verbose=False, networkx=False)[source]

Create plots of module dependencies.

Parameters
  • srcdir (str) – path for source files

  • outdir (str) – path for output images

  • include_subdir (bool) – boolean indicating is subdirectories in the source file directory should be included

  • level (int) – dependency level (1 is the minimum)

  • extension (str) – output extension (default is .png)

  • verbose (bool) – boolean indicating if output will be printed to the terminal

  • networkx (bool) – boolean indicating that the NetworkX python package will be used to create the Directed Acyclic Graph (DAG) used to determine the order source files are compiled in. The NetworkX package tends to result in a unique DAG more often than the standard algorithm used in pymake. (default is False)

parser(examples=None)[source]

Construct the parser and return argument values.

Parameters

examples (str) –

Returns

args – Namespace with command line arguments

Return type

Namespace object

repo_latest_version(github_repo=None, verify=True)[source]

Return a string of the latest version number (tag) contained in a github repository release.

Parameters

github_repo (str) – Repository name, such as MODFLOW-USGS/modflow6. If github_repo is None set to ‘MODFLOW-USGS/executables’

Returns

version – string with the latest version/tag number

Return type

str

setup(namefile, dst, remove_existing=True, extrafiles=None)[source]

Setup MODFLOW-based model files for autotests.

Parameters
  • namefile (str) – MODFLOW-based model name file.

  • dst (str) – destination path for comparison model or file(s)

  • remove_existing (bool) – boolean indicating if an existing comparision model or file(s) should be replaced (default is True)

  • extrafiles (str or list of str) – list of extra files to include in the comparision

setup_comparison(namefile, dst, remove_existing=True)[source]

Setup a comparison model or comparision file(s) for a MODFLOW-based model.

Parameters
  • namefile (str) – MODFLOW-based model name file.

  • dst (str) – destination path for comparison model or file(s)

  • remove_existing (bool) – boolean indicating if an existing comparision model or file(s) should be replaced (default is True)

setup_mf6(src, dst, mfnamefile='mfsim.nam', extrafiles=None, remove_existing=True)[source]

Copy all of the MODFLOW 6 input files from the src directory to the dst directory.

Parameters
  • src (src) – directory path with original MODFLOW 6 input files

  • dst (str) – directory path that original MODFLOW 6 input files will be copied to

  • mfnamefile (str) – optional MODFLOW 6 simulation name file (default is mfsim.nam)

  • extrafiles (bool) – boolean indicating if extra files should be included (default is None)

  • remove_existing (bool) – boolean indicating if existing file in dst should be removed (default is True)

Returns

  • mf6inp (list) – list of MODFLOW 6 input files

  • mf6outp (list) – list of MODFLOW 6 output files

setup_mf6_comparison(src, dst, remove_existing=True)[source]

Setup comparision for MODFLOW 6 simulation.

Parameters
  • src (src) – directory path with original MODFLOW 6 input files

  • dst (str) – directory path that original MODFLOW 6 input files will be copied to

  • remove_existing (bool) – boolean indicating if existing file in dst should be removed (default is True)

Returns

action – comparison type

Return type

str

teardown(src)[source]

Teardown a autotest directory.

Parameters

src (str) – autotest directory to teardown

to_pydot(dag, filename='mygraph.png')[source]

Create a png file of a Directed Acyclic Graph

Parameters
  • dag (object) – directed acyclic graph

  • filename (str) – path of the graph png

class usgs_program_data[source]

Bases: object

USGS program database class.

static export_json(fpth='code.json', prog_data=None, current=False, update=True, write_markdown=False, verbose=False)[source]

Export USGS program data as a json file.

Parameters
  • fpth (str) – Path for the json file to be created. Default is “code.json”

  • prog_data (dict) – User-specified program database. If prog_data is None, it will be created from the USGS program database

  • current (bool) – If False, all USGS program targets are listed. If True, only USGS program targets that are defined as current are listed. Default is False.

  • update (bool) – If True, existing targets in the user-specified program database with values in the USGS program database. If False, existing targets in the user-specified program database will not be updated. Default is True.

  • write_markdown (bool) – If True, write markdown file that includes the target name, version, and the last-modified date of the download asset (url). Default is False.

  • verbose (bool) – boolean for verbose output to terminal

static get_keys(current=False)[source]

Get target keys from the USGS program database.

Parameters

current (bool) – If False, all USGS program targets are listed. If True, only USGS program targets that are defined as current are listed. Default is False.

Returns

keys – list of USGS program targets

Return type

list

static get_precision(key)[source]

Get the dictionary for a specified target.

Parameters

key (str) – Target USGS program

Returns

precision – List

Return type

list

static get_program_dict()[source]

Get the complete USGS program database.

Returns

program_dict – Dictionary with USGS program attributes for all targets

Return type

dict

static get_target(key)[source]

Get the dictionary for a specified target.

Parameters

key (str) – Target USGS program that may have a path and an extension

Returns

program_dict – Dictionary with USGS program attributes for the specified key

Return type

dict

static get_version(key)[source]

Get the current version of the specified target.

Parameters

key (str) – Target USGS program

Returns

version – current version of the specified target

Return type

str

static list_json(fpth='code.json')[source]

List an existing code json file.

Parameters

fpth (str) – Path for the json file to be listed. Default is “code.json”

static list_targets(current=False)[source]

Print a list of the available USGS program targets.

Parameters

current (bool) – If False, all USGS program targets are listed. If True, only USGS program targets that are defined as current are listed. Default is False.

static load_json(fpth='code.json')[source]

Load an existing code json file. Basic error checking is done to make sure the file contains the correct keys.

Parameters

fpth (str) – Path for the json file to be created. Default is “code.json”

Returns

json_dict – Valid USGS program database

Return type

dict

static update_json(fpth='code.json', temp_dict=None)[source]

UPDATE an existing code json file.

Parameters
  • fpth (str) – Path for the json file to be listed. Default is “code.json”

  • temp_dict (dict) – Dictionary with USGS program data for a target

zip_all(path, file_pths=None, dir_pths=None, patterns=None)[source]

Compress all files in the user-provided list of file paths and directory paths that match the provided file patterns.

Parameters
  • path (str) – path of the zip file that will be created

  • file_pths (str or list) – file path or list of file paths to be compressed

  • dir_pths (str or list) – directory path or list of directory paths to search for files that will be compressed

  • patterns (str or list) – file pattern or list of file patterns s to match to when creating a list of files that will be compressed