pymake Documentation¶
This is a python package for compiling MODFLOW-based and other Fortran, C, and
C++ programs. The package determines the build order using a directed acyclic
graph and then compiles the source files using GNU compilers (gcc,
g++, gfortran) or Intel compilers (ifort, icc).
pymake can be run from the command line or it can be called from within python. By default, pymake sets the optimization level, Fortran flags, C/C++ flags, and linker flags that are consistent with those used to compile MODFLOW-based programs released by the USGS.
pymake includes example scripts for building MODFLOW 6, MODFLOW-2005, MODFLOW-NWT, MODFLOW-USG, MODFLOW-LGR, MODFLOW-2000, MODPATH 6, MODPATH 7, GSFLOW, VS2DT, MT3DMS, MT3D-USGS, SEAWAT, and SUTRA. Example scripts for creating the utility programs CRT, Triangle, and GRIDGEN are also included. The scripts download the distribution file from the USGS (and other organizations) and compile the source into a binary executable.
The main documentation for the site is organized into the following sections:
Getting Started¶
Pymake can be used directly from the command line and within a python script. Examples of both use cases are given below.
Command Line Usage¶
When pymake is installed, a mfpymake (or mfpymake.exe for Windows) program is installed. mfpymake can be used to
compile MODFLOW 6 in the bin/ subdirectory directly from the command line using
the Intel Fortran compiler from a subdirectory at the same level as
the src subdirectory by specifying:
mfpymake src/ mf6 -mc --subdirs -fc ifort --appdir bin
MODFLOW 6 would be To see help for running from command line, use the following statement.
mfpymake -h
The help message identifies required positional arguments and optional arguments that can be provided to overide default values.
usage: mfpymake [-h] [-fc {ifort,mpiifort,gfortran,ftn,none}]
[-cc {gcc,clang,clang++,icc,icl,mpiicc,g++,cl,none}]
[-ar {ia32,ia32_intel64,intel64}] [-mc] [-dbl] [-dbg]
[-e] [-dr] [-sd] [-ff FFLAGS] [-cf CFLAGS]
[-sl {-lc,-lm}] [-mf] [-md] [-cs COMMONSRC]
[-ef EXTRAFILES] [-exf EXCLUDEFILES] [-so]
[-ad APPDIR] [-v] [--keep] [--zip ZIP] [--inplace]
[--networkx] [--mb] [-mbd]
srcdir target
This is the pymake program for compiling fortran, c, and c++ source
files, such as the source files that come with MODFLOW. The program
works by building a directed acyclic graph of the module dependencies
and then compiling the source files in the proper order.
positional arguments:
srcdir Path source directory.
target Name of target to create. (can include path)
optional arguments:
-h, --help show this help message and exit
-fc {ifort,mpiifort,gfortran,ftn,none}
Fortran compiler to use. (default is
gfortran)
-cc {gcc,clang,clang++,icc,icl,mpiicc,g++,cl,none}
C/C++ compiler to use. (default is gcc)
-ar {ia32,ia32_intel64,intel64}, --arch {ia32,ia32_intel64,intel64}
Architecture to use for Intel and Microsoft
compilers on Windows. (default is intel64)
-mc, --makeclean Clean temporary object, module, and source
files when done. (default is False)
-dbl, --double Force double precision. (default is False)
-dbg, --debug Create debug version. (default is False)
-e, --expedite Only compile out of date source files. Clean
must not have been used on previous build.
(default is False)
-dr, --dryrun Do not actually compile. Files will be
deleted, if --makeclean is used. Does not
work yet for ifort. (default is False)
-sd, --subdirs Include source files in srcdir
subdirectories. (default is None)
-ff FFLAGS, --fflags FFLAGS
Additional Fortran compiler flags. Fortran
compiler flags should be enclosed in quotes
and start with a blank space or separated
from the name (-ff or --fflags) with a equal
sign (-ff='-O3'). (default is None)
-cf CFLAGS, --cflags CFLAGS
Additional C/C++ compiler flags. C/C++
compiler flags should be enclosed in quotes
and start with a blank space or separated
from the name (-cf or --cflags) with a equal
sign (-cf='-O3'). (default is None)
-sl {-lc,-lm}, --syslibs {-lc,-lm}
Linker system libraries. Linker libraries
should be enclosed in quotes and start with a
blank space or separated from the name (-sl
or --syslibs) with a equal sign
(-sl='-libgcc'). (default is None)
-mf, --makefile Create a GNU make makefile. (default is
False)
-md, --makefile-dir GNU make makefile directory. (default is '.')
-cs COMMONSRC, --commonsrc COMMONSRC
Additional directory with common source
files. (default is None)
-ef EXTRAFILES, --extrafiles EXTRAFILES
List of extra source files to include in the
compilation. extrafiles can be either a list
of files or the name of a text file that
contains a list of files. (default is None)
-exf EXCLUDEFILES, --excludefiles EXCLUDEFILES
List of extra source files to exclude from
the compilation. excludefiles can be either a
list of files or the name of a text file that
contains a list of files. (default is None)
-so, --sharedobject Create shared object or dll on Windows.
(default is False)
-ad APPDIR, --appdir APPDIR
Target path that overides path defined target
path (default is None)
-v, --verbose Verbose output to terminal. (default is
False)
--keep Keep existing executable. (default is False)
--zip ZIP Zip built executable. (default is False)
--inplace Source files in srcdir are used directly.
(default is False)
--networkx Use networkx package to build Directed
Acyclic Graph use to determine the order
source files are compiled in. (default is
False)
--mb, --meson-build Use meson to build executable. (default is
False)
-mbd, --mesonbuild-dir
meson directory. (default is '.')
Note that the source directory should not contain any bad
or duplicate source files as all source files in the source
directory, the common source file directory (srcdir2), and
the extra files (extrafiles) will be built and linked.
Files can be excluded by using the excludefiles command
line switch.
Examples:
Compile MODFLOW 6 from the root directory containing the
source files in subdirectories in the src/ subdirectory:
$ mfpymake src/ mf6 --subdirs
Compile MODFLOW 6 in the bin subdirectory using the Intel
Fortran compiler from the root directory containing the source
files in subdirectories in the the src/ subdirectory:
$ mfpymake src/ mf6 --subdirs -fc ifort --appdir bin
Note that command line arguments for Fortran flags, C/C++ flags, and syslib
libraries should be enclosed in quotes and start with a space prior to the
first value (-ff ' -O3') or use an equal sign separating the command line
argument and the values (-ff='-O3'). The command line argument to use an
-O3 optimization level when compiling MODFLOW 6 in the bin/ subdirectory with the Intel Fortran compiler
would be:
mfpymake ../src/ mf6 -mc --subdirs -fc ifort -ff='-O3' --appdir bin
From Python¶
Script to compile MODFLOW 6¶
When using the pymake object (Pymake()) only the positional arguments
(srcdir, target) need to be specified in the script.
import pymake
pm = pymake.Pymake()
pm.srcdir = '../src'
pm.target = 'mf6'
pm.include_subdirs = True
pm.build()
It is suggested that optional variables required for successful compiling and
linking be manually specified in the script to mininimize the potential for
unsuccessful builds. For MODFLOW 6, subdirectories in the src subdirectory
need to be included and ‘pm.include_subdirs = True’ has been specified in
the script. Custom optimization levels and compiler flags could be specified
to get consistent builds.
Non-default values for the optional arguments can specified as command line arguments. For example, MODFLOW 6 could be compiled using Intel compilers instead of the default GNU compilers with the script listed above by specifying:
python mymf6script.py -fc ifort -cc icc
Notes¶
If gfortran is used to compile MODFLOW-based codes, the openspec.f
and FILESPEC.inc (MT3DMS) files will automatically be changed to the
following so that binary files are created properly using standard Fortran:
c -- created by pymake.py
CHARACTER*20 ACCESS,FORM,ACTION(2)
DATA ACCESS/'STREAM'/
DATA FORM/'UNFORMATTED'/
DATA (ACTION(I),I=1,2)/'READ','READWRITE'/
c -- end of include file
Use of STREAM access does not delete an existing unformatted file before
opening it for writing. As a result, data from previous runs may exist in the
file if the model is run for a shorter period of time. This does not apply to
MODFLOW 6 simulations.
pymake Installation¶
To install a stable version from PyPI:
pip install mfpymake
To install pymake directly from the git repository type:
pip install https://github.com/modflowpy/pymake/zipball/master
To update your version of pymake with the latest from the git repository type:
pip install https://github.com/modflowpy/pymake/zipball/master --upgrade
Building Applications¶
When pymake is installed, a make-program (or make-program.exe for Windows) program is installed, which is usually
installed to a directory in the PATH (depending on the Python setup). From a console a list of command line arguments
and options can be determined by executing:
$ make-program --help
usage: make-program [-h] [--release_precision]
[-fc {ifort,mpiifort,gfortran,ftn,none}]
[-cc {gcc,clang,clang++,icc,icx,icl,mpiicc,g++,cl,none}]
[-dr] [-ff FFLAGS] [-cf CFLAGS] [-ad APPDIR] [-v] [--keep]
[--zip ZIP]
targets
Download and build USGS MODFLOW and related programs.
positional arguments:
targets Program(s) to build. Options: crt, gridgen, gsflow,
libmf6, mf2000, mf2005, mf6, mflgr, mfnwt, mfusg, mp6,
mp7, mt3dms, mt3dusgs, prms, sutra, swtv4, triangle,
vs2dt, zbud6, zonbud3, zonbudusg, :. Specifying the
target to be ':' will build all of the programs.
optional arguments:
-h, --help show this help message and exit
--release_precision If release_precision is False, then the release
precision version will be compiled along with a double
precision version of the program for programs where
the standard_switch and double_switch in
usgsprograms.txt is True. default is True.
-fc {ifort,mpiifort,gfortran,ftn,none}
Fortran compiler to use. (default is gfortran)
-cc {gcc,clang,clang++,icc,icx,icl,mpiicc,g++,cl,none}
C/C++ compiler to use. (default is gcc)
-dr, --dryrun Do not actually compile. Files will be deleted, if
--makeclean is used. Does not work yet for ifort.
(default is False)
-ff FFLAGS, --fflags FFLAGS
Additional Fortran compiler flags. Fortran compiler
flags should be enclosed in quotes and start with a
blank space or separated from the name (-ff or
--fflags) with a equal sign (-ff='-O3'). (default is
None)
-cf CFLAGS, --cflags CFLAGS
Additional C/C++ compiler flags. C/C++ compiler flags
should be enclosed in quotes and start with a blank
space or separated from the name (-cf or --cflags)
with a equal sign (-cf='-O3'). (default is None)
-ad APPDIR, --appdir APPDIR
Target path that overides path defined target path
(default is None)
-v, --verbose Verbose output to terminal. (default is False)
--keep Keep existing executable. (default is False)
--zip ZIP Zip built executable. (default is False)
Examples:
Download and compile MODFLOW 6 in the current directory:
$ make-program mf6
Download and compile triangle in the ./temp subdirectory:
$ make-program triangle --appdir temp
Download and compile all programs in the ./temp subdirectory:
$ make-program : --appdir temp
make-program can be used to build MODFLOW 6, MODFLOW-2005, MODFLOW-NWT, MODFLOW-USG, MODFLOW-LGR, MODFLOW-2000,
MODPATH 6, MODPATH 7, GSFLOW, VS2DT, MT3DMS, MT3D-USGS, SEAWAT, GSFLOW, PRMS, and SUTRA. Utility programs CRT, Triangle,
and GRIDGEN can also be built. make-program downloads the distribution file from the USGS (requires internet
connection), unzips the distribution file, sets the pymake settings required to build the program, and compiles the
source files to build the program. MT3DMS will be downloaded from the University of Alabama and Triangle will be
downloaded from netlib.org. Optional command line arguments can be used to
customize the build (-fc, -cc, --fflags, etc.). For example, MODFLOW 6 could be built using intel compilers and
an O3 optimation level by specifying:
make-program mf6 -fc=ifort --fflags='-O3'
API-docs¶
This section contains the Documentation of the Application Programming Interface (API) of pymake. The information in this section is automatically created from the documentation strings in original Python code. In the left-hand menu you will find the different categories of the API documentation.
pymake.pymake module¶
Pymake() class to make a binary executable for a FORTRAN, C, or C++
program, such as MODFLOW 6.
An example of how to build MODFLOW-2005 from source files in the official release downloaded from the USGS using Intel compilers is:
import pymake
# create an instance of the Pymake object
pm = pymake.Pymake(verbose=True)
# reset select pymake settings
pm.target = "mf2005"
pm.appdir = "../bin"
pm.fc = "ifort"
pm.cc = "icc"
pm.fflags = "-O3 -fbacktrace"
pm.cflags = "-O3"
# download the target
pm.download_target(pm.target, download_path="temp")
# build the target
pm.build()
# clean up downloaded files
pm.finalize()
All other settings not specified in the script would be based on command
line arguments or default values. The same Pymake() object could be
used to compile MODFLOW 6 by appending the following code to the previous code
block:
# reset the target
pm.target = "mf6"
# download the target
pm.download_target(pm.target, download_path="temp")
# build the target
pm.build()
# clean up downloaded files
pm.finalize()
The Intel compilers and fortran flags defined previously would be used when MODFLOW 6 was built.
- class Pymake(name='pymake', verbose=None)[source]¶
Bases:
objectPymake class for interacting with pymake functionality. This is essentially a wrapper for all of the pymake functions needed to download and build a target.
- argv_reset_settings(args)[source]¶
Reset settings using command line arguments
- Parameters
args (Namespace object) – reset self.variables using command line arguments
- build(target=None, srcdir=None, modify_exe_name=False)[source]¶
Build the target
- Parameters
target (str) – target name. If target is None self.target is used. (default is None)
srcdir (str) – path to directory with source files. (default is None)
modify_exe_name (bool) – boolean that determines if the target name can be modified to include precision (dbl) and debugging (d) indicators.
- download_setup(target, url=None, download_path='.', verify=True, timeout=30)[source]¶
Setup download
- Parameters
target (str) – target name
url (str) – url of asset
download_path (str) – path where the asset will be saved
verify (bool) – boolean defining ssl verification
timeout (int) – download timeout in seconds (default is 30)
- download_target(target, url=None, download_path='.', verify=True, timeout=30)[source]¶
Setup and download url
- Parameters
target (str) – target name
url (str) – url of asset
download_path (str) – path where the asset will be saved
verify (bool) – boolean defining ssl verification
timeout (int) – download timeout in seconds (default is 30)
- Returns
success – boolean flag indicating download success
- Return type
bool
- download_url()[source]¶
Download files from the url
- Returns
success – boolean flag indicating download success
- Return type
bool
- reset(target)[source]¶
Reset PyMake object variables for a target
- Parameters
target (str) – target name
- set_build_target_bool(target=None)[source]¶
Evaluate if the executable exists and if so and the command line argument –keep is specified then the executable is not built.
- Parameters
target (str) – target name. If target is None self.target will be used. (default is None)
- Returns
build – boolean indicating if the executable should be built
- Return type
bool
- update_target(target, modify_target=False)[source]¶
Update target name with executable extension on Windows and based on pymake settings.
- Parameters
target (str) – target name
modify_target (bool) – boolean indicating if the target name can be modified based on pymake double and debug settings (default is False)
- Returns
target – updated target name
- Return type
str
pymake.pymake_build_apps module¶
Function to build MODFLOW-based models and other utility software based on
targets defined in the usgsprograms database (usgsprograms.txt). The
usgsprograms database can be queried using functions in the usgsprograms
module. An example of using pymake.build_apps() to build MODFLOW 6 is:
import pymake
pymake.build_apps(["mf6",])
which will download the latest MODFLOW 6 software release, compile the code,
and delete the downloaded files after successfully building the application.
Multiple applications can be built by adding additional targets to the tuple
in pymake.build_apps(). For example, MODFLOW 6 and MODFLOW-2005 could
be built by specifying:
import pymake
pymake.build_apps(["mf6","mf2005"]))
Applications are built in the order they are listed in the list. All valid
USGS applications are built if no list is passed to
pymake.build_apps().
- build_apps(targets=None, pymake_object=None, download_dir=None, appdir=None, verbose=None, release_precision=True, meson=False, mesondir='.', clean=True)[source]¶
Build all of the current targets or a subset of targets.
- Parameters
targets (str or list of str) – targets to build. If targets is None, all current targets will be build. Default is None
pymake_object (Pymake()) – Pymake object created outside of build_apps
download_dir (str) – download directory path
appdir (str) – target path
release_precision (bool) – boolean indicating if only the release precision version should be build. If release_precision is False, then the release precision version will be compiled along with a double precision version of the program for programs where the standard_switch and double_switch in usgsprograms.txt is True. default is True.
meson (bool) – boolean indicating that the executable should be built using the meson build system. (default is False)
mesondir (str) – Main meson.build file path
clean (bool) – boolean determining of final download should be removed
- Returns
returncode – integer value indicating successful completion (0) or failure (>0)
- Return type
int
pymake.pymake_base module¶
Main pymake function, pymake.main(), that is called when pymake is
run from the command line. pymake.main() can also be called directly
from a script in combination with pymake.parser().
import pymake
args = pymake.parser()
pymake.main(
args.srcdir,
args.target,
fc=args.fc,
cc=args.cc,
makeclean=args.makeclean,
expedite=args.expedite,
dryrun=args.dryrun,
double=args.double,
debug=args.debug,
include_subdirs=args.subdirs,
fflags=args.fflags,
cflags=args.cflags,
arch=args.arch,
syslibs=args.syslibs,
makefile=args.makefile,
srcdir2=args.commonsrc,
extrafiles=args.extrafiles,
excludefiles=args.excludefiles,
sharedobject=args.sharedobject,
appdir=args.appdir,
verbose=args.verbose,
inplace=args.inplace,
)
The script could be run from the command line using:
python myscript.py ../src myapp -fc=ifort -cc=icc
- get_temporary_directories(appdir=None, target=None)[source]¶
Get paths to temporary object, module, and source files
- Parameters
appdir (str) – path for executable
target (str) – target name to be appended to the temporary directories. Default is None
- Returns
obj_temp (str) – path to temporary object files
mod_temp (str) – path to temporary module files
src_temp (str) – path to temporary source files
- main(srcdir=None, target=None, fc='gfortran', cc='gcc', makeclean=True, expedite=False, dryrun=False, double=False, debug=False, include_subdirs=False, fflags=None, cflags=None, syslibs=None, arch='intel64', makefile=False, makefiledir='.', srcdir2=None, extrafiles=None, excludefiles=None, sharedobject=False, appdir=None, verbose=False, inplace=False, networkx=False, meson=False, mesondir='.')[source]¶
Main pymake function.
- Parameters
srcdir (str) – path for directory containing source files
target (str) – executable name or path for executable to create
fc (str) – fortran compiler
cc (str) – c or cpp compiler
makeclean (bool) – boolean indicating if intermediate files should be cleaned up after successful build
expedite (bool) – boolean indicating if only out of date source files will be compiled. Clean must not have been used on previous build.
dryrun (bool) – boolean indicating if source files should be compiled. Files will be deleted, if makeclean is True.
double (bool) – boolean indicating a compiler switch will be used to create an executable with double precision real variables.
debug (bool) – boolean indicating is a debug executable will be built
include_subdirs (bool) – boolean indicating source files in srcdir subdirectories should be included in the build
fflags (list) – user provided list of fortran compiler flags
cflags (list) – user provided list of c or cpp compiler flags
syslibs (list) – user provided syslibs
arch (str) – Architecture to use for Intel Compilers on Windows (default is intel64)
makefile (bool) – boolean indicating if a GNU make makefile should be created
makefiledir (str) – GNU make makefile path
srcdir2 (str) – additional directory with common source files.
extrafiles (str) – path for extrafiles file that contains paths to additional source files to include
excludefiles (str) – path for excludefiles file that contains filename of source files to exclude from the build
sharedobject (bool) – boolean indicating a shared object will be built
appdir (str) – path for executable
verbose (bool) – boolean indicating if output will be printed to the terminal
inplace (bool) – boolean indicating that the source files in srcdir, srcdir2, and defined in extrafiles will be used directly. If inplace is False, source files will be copied to a directory named srcdir_temp. (default is False)
networkx (bool) – boolean indicating that the NetworkX python package will be used to create the Directed Acyclic Graph (DAG) used to determine the order source files are compiled in. The NetworkX package tends to result in a unique DAG more often than the standard algorithm used in pymake. (default is False)
meson (bool) – boolean indicating that the executable should be built using the meson build system. (default is False)
mesondir (str) – Main meson.build file path
- Returns
returncode – return code
- Return type
int
pymake.pymake_parser module¶
Parser used to process command line arguments when running pymake directly from the command line or in a script. The standard argparse module is used to parse command line arguments. Available command line arguments are programmatically developed by a protected dictionary. The parser can be accessed using:
import pymake
args = pymake.parser()
pymake.plot package¶
Submodules¶
pymake.plot.dependency_graphs module¶
Dependency graphs for applications can be created using:
import os
import pymake
srcpth = os.path.join("..", "src")
deppth = "dependencies"
if not os.path.exists(deppth):
os.makedirs(deppth)
pymake.visualize.make_plots(srcpth, deppth, include_subdir=True)
- make_plots(srcdir, outdir, include_subdir=False, level=3, extension='.png', verbose=False, networkx=False)[source]¶
Create plots of module dependencies.
- Parameters
srcdir (str) – path for source files
outdir (str) – path for output images
include_subdir (bool) – boolean indicating is subdirectories in the source file directory should be included
level (int) – dependency level (1 is the minimum)
extension (str) – output extension (default is .png)
verbose (bool) – boolean indicating if output will be printed to the terminal
networkx (bool) – boolean indicating that the NetworkX python package will be used to create the Directed Acyclic Graph (DAG) used to determine the order source files are compiled in. The NetworkX package tends to result in a unique DAG more often than the standard algorithm used in pymake. (default is False)
Module contents¶
Functions to plot source code dependencies determined using a directed acyclic graph (DAG). pydotplus is used to plot the DAG.
pymake.autotest package¶
Submodules¶
pymake.autotest.autotest module¶
A typical example of using the autotest functionality for MODFLOW-2005 and comparing the MODFLOW-2005 results to MODFLOW-2000 results is:
import pymake
# Setup
testpth = "../test/mytest"
nam1 = "model1.nam"
pymake.setup(nam1, testpth)
# run test models
success, buff = flopy.run_model(
"mf2005", nam1, model_ws=testpth, silent=True
)
if success:
testpth_reg = os.path.join(testpth, "mf2000")
nam2 = "model2.name"
pymake.setup(nam2, testpth_reg)
success_reg, buff = flopy.run_model(
"mf2000", nam2, model_ws=testpth_reg, silent=True
)
# compare results
if success and success_reg:
fpth = os.path.split(os.path.join(testpth, nam1))[0]
outfile1 = os.path.join(fpth, "bud.cmp")
fpth = os.path.split(os.path.join(testpth, nam2))[0]
outfile2 = os.path.join(fpth, "hds.cmp")
success_reg = pymake.compare(
os.path.join(testpth, nam1),
os.path.join(testpth_reg, nam2),
max_cumpd=0.01,
max_incpd=0.01,
htol=0.001,
outfile1=outfile1,
outfile2=outfile2,
)
# Clean things up
if success_reg:
pymake.teardown(testpth)
Note: autotest functionality will likely be removed from pymake in the future to a dedicated GitHub repository.
- compare(namefile1, namefile2, precision='auto', max_cumpd=0.01, max_incpd=0.01, htol=0.001, outfile1=None, outfile2=None, files1=None, files2=None)[source]¶
Compare the budget and head results for two MODFLOW-based model simulations.
- Parameters
namefile1 (str) – namefile path for base model
namefile2 (str) – namefile path for comparison model
precision (str) – precision for binary head file (“auto”, “single”, or “double”) default is “auto”
max_cumpd (float) – maximum percent discrepancy allowed for cumulative budget terms (default is 0.01)
max_incpd (float) – maximum percent discrepancy allowed for incremental budget terms (default is 0.01)
htol (float) – maximum allowed head difference (default is 0.001)
outfile1 (str) – budget comparison output file name. If outfile1 is None, no budget comparison output is saved. (default is None)
outfile2 (str) – head comparison output file name. If outfile2 is None, no head comparison output is saved. (default is None)
files1 (str) – base model output file. If files1 is not None, results will be extracted from files1 and namefile1 will not be used. (default is None)
files2 (str) – comparison model output file. If files2 is not None, results will be extracted from files2 and namefile2 will not be used. (default is None)
- Returns
success – boolean indicating if the budget and head differences are less than max_cumpd, max_incpd, and htol.
- Return type
bool
- compare_budget(namefile1, namefile2, max_cumpd=0.01, max_incpd=0.01, outfile=None, files1=None, files2=None)[source]¶
Compare the budget results from two simulations.
- Parameters
namefile1 (str) – namefile path for base model
namefile2 (str) – namefile path for comparison model
max_cumpd (float) – maximum percent discrepancy allowed for cumulative budget terms (default is 0.01)
max_incpd (float) – maximum percent discrepancy allowed for incremental budget terms (default is 0.01)
outfile (str) – budget comparison output file name. If outfile is None, no comparison output is saved. (default is None)
files1 (str) – base model output file. If files1 is not None, results will be extracted from files1 and namefile1 will not be used. (default is None)
files2 (str) – comparison model output file. If files2 is not None, results will be extracted from files2 and namefile2 will not be used. (default is None)
- Returns
success – boolean indicating if the difference between budgets are less than max_cumpd and max_incpd
- Return type
bool
- compare_concs(namefile1, namefile2, precision='auto', ctol=0.001, outfile=None, files1=None, files2=None, difftol=False, verbose=False)[source]¶
Compare the mt3dms and mt3dusgs concentration results from two simulations.
- Parameters
namefile1 (str) – namefile path for base model
namefile2 (str) – namefile path for comparison model
precision (str) – precision for binary head file (“auto”, “single”, or “double”) default is “auto”
ctol (float) – maximum allowed concentration difference (default is 0.001)
outfile (str) – concentration comparison output file name. If outfile is None, no comparison output is saved. (default is None)
files1 (str) – base model output file. If files1 is not None, results will be extracted from files1 and namefile1 will not be used. (default is None)
files2 (str) – comparison model output file. If files2 is not None, results will be extracted from files2 and namefile2 will not be used. (default is None)
difftol (bool) – boolean determining if the absolute value of the concentration difference greater than ctol should be evaluated (default is False)
verbose (bool) – boolean indicating if verbose output should be written to the terminal (default is False)
- Returns
success – boolean indicating if the concentration differences are less than ctol.
- Return type
bool
- compare_heads(namefile1, namefile2, precision='auto', text='head', text2=None, htol=0.001, outfile=None, files1=None, files2=None, difftol=False, verbose=False, exfile=None, exarr=None, maxerr=None)[source]¶
Compare the head results from two simulations.
- Parameters
namefile1 (str) – namefile path for base model
namefile2 (str) – namefile path for comparison model
precision (str) – precision for binary head file (“auto”, “single”, or “double”) default is “auto”
htol (float) – maximum allowed head difference (default is 0.001)
outfile (str) – head comparison output file name. If outfile is None, no comparison output is saved. (default is None)
files1 (str) – base model output file. If files1 is not None, results will be extracted from files1 and namefile1 will not be used. (default is None)
files2 (str) – comparison model output file. If files2 is not None, results will be extracted from files2 and namefile2 will not be used. (default is None)
difftol (bool) – boolean determining if the absolute value of the head difference greater than htol should be evaluated (default is False)
verbose (bool) – boolean indicating if verbose output should be written to the terminal (default is False)
exfile (str) – path to a file with exclusion array data. Head differences will not be evaluated where exclusion array values are greater than zero. (default is None)
exarr (numpy.ndarry) – exclusion array. Head differences will not be evaluated where exclusion array values are greater than zero. (default is None).
maxerr (int) – maximum number of head difference greater than htol that should be reported. If maxerr is None, all head difference greater than htol will be reported. (default is None)
- Returns
success – boolean indicating if the head differences are less than htol.
- Return type
bool
- compare_stages(namefile1=None, namefile2=None, files1=None, files2=None, htol=0.001, outfile=None, difftol=False, verbose=False)[source]¶
Compare SWR process stage results from two simulations.
- Parameters
namefile1 (str) – namefile path for base model
namefile2 (str) – namefile path for comparison model
precision (str) – precision for binary head file (“auto”, “single”, or “double”) default is “auto”
htol (float) – maximum allowed stage difference (default is 0.001)
outfile (str) – head comparison output file name. If outfile is None, no comparison output is saved. (default is None)
files1 (str) – base model output file. If files1 is not None, results will be extracted from files1 and namefile1 will not be used. (default is None)
files2 (str) – comparison model output file. If files2 is not None, results will be extracted from files2 and namefile2 will not be used. (default is None)
difftol (bool) – boolean determining if the absolute value of the stage difference greater than htol should be evaluated (default is False)
verbose (bool) – boolean indicating if verbose output should be written to the terminal (default is False)
- Returns
success – boolean indicating if the stage differences are less than htol.
- Return type
bool
- compare_swrbudget(namefile1, namefile2, max_cumpd=0.01, max_incpd=0.01, outfile=None, files1=None, files2=None)[source]¶
Compare the SWR budget results from two simulations.
- Parameters
namefile1 (str) – namefile path for base model
namefile2 (str) – namefile path for comparison model
max_cumpd (float) – maximum percent discrepancy allowed for cumulative budget terms (default is 0.01)
max_incpd (float) – maximum percent discrepancy allowed for incremental budget terms (default is 0.01)
outfile (str) – budget comparison output file name. If outfile is None, no comparison output is saved. (default is None)
files1 (str) – base model output file. If files1 is not None, results will be extracted from files1 and namefile1 will not be used. (default is None)
files2 (str) – comparison model output file. If files2 is not None, results will be extracted from files2 and namefile2 will not be used. (default is None)
- Returns
success – boolean indicating if the difference between budgets are less than max_cumpd and max_incpd
- Return type
bool
- get_entries_from_namefile(namefile, ftype=None, unit=None, extension=None)[source]¶
Get entries from a namefile. Can select using FTYPE, UNIT, or file extension.
- Parameters
namefile (str) – path to a MODFLOW-based model name file
ftype (str) – package type
unit (int) – file unit number
extension (str) – file extension
- Returns
entries – list of tuples containing FTYPE, UNIT, FNAME, STATUS for each namefile entry that meets a user-specified value.
- Return type
list of tuples
- get_input_files(namefile)[source]¶
Return a list of all the input files in this model.
- Parameters
namefile (str) – path to a MODFLOW-based model name file
- Returns
filelist – list of MODFLOW-based model input files
- Return type
list
- get_mf6_blockdata(f, blockstr)[source]¶
Return list with all non comments between start and end of block specified by blockstr.
- Parameters
f (file object) – open file object
blockstr (str) – name of block to search
- Returns
data – list of data in specified block
- Return type
list
- get_mf6_comparison(src)[source]¶
Determine comparison type for MODFLOW 6 simulation.
- Parameters
src (str) – directory path to search for comparison types
- Returns
action – comparison type
- Return type
str
- get_mf6_files(mfnamefile)[source]¶
Return a list of all the MODFLOW 6 input and output files in this model.
- Parameters
mfnamefile (str) – path to the MODFLOW 6 simulation name file
- Returns
filelist (list) – list of MODFLOW 6 input files in a simulation
outplist (list) – list of MODFLOW 6 output files in a simulation
- get_mf6_ftypes(namefile, ftypekeys)[source]¶
Return a list of FTYPES that are in the name file and in ftypekeys.
- Parameters
namefile (str) – path to a MODFLOW 6 name file
ftypekeys (list) – list of desired FTYPEs
- Returns
ftypes – list of FTYPES that match ftypekeys in namefile
- Return type
list
- get_mf6_mshape(disfile)[source]¶
Return the shape of the MODFLOW 6 model.
- Parameters
disfile (str) – path to a MODFLOW 6 discretization file
- Returns
mshape – tuple with the shape of the MODFLOW 6 model.
- Return type
tuple
- get_mf6_nper(tdisfile)[source]¶
Return the number of stress periods in the MODFLOW 6 model.
- Parameters
tdisfile (str) – path to the TDIS file
- Returns
nper – number of stress periods in the simulation
- Return type
int
- get_namefiles(pth, exclude=None)[source]¶
Search through a path (pth) for all .nam files.
- Parameters
pth (str) – path to model files
exclude (str or lst) – File or list of files to exclude from the search (default is None)
- Returns
namefiles – List of namefiles with paths
- Return type
lst
- get_sim_name(namefiles, rootpth=None)[source]¶
Get simulation name.
- Parameters
namefiles (str or list of strings) – path(s) to MODFLOW-based model name files
rootpth (str) – optional root directory path (default is None)
- Returns
simname – list of namefiles without the file extension
- Return type
list
- setup(namefile, dst, remove_existing=True, extrafiles=None)[source]¶
Setup MODFLOW-based model files for autotests.
- Parameters
namefile (str) – MODFLOW-based model name file.
dst (str) – destination path for comparison model or file(s)
remove_existing (bool) – boolean indicating if an existing comparision model or file(s) should be replaced (default is True)
extrafiles (str or list of str) – list of extra files to include in the comparision
- setup_comparison(namefile, dst, remove_existing=True)[source]¶
Setup a comparison model or comparision file(s) for a MODFLOW-based model.
- Parameters
namefile (str) – MODFLOW-based model name file.
dst (str) – destination path for comparison model or file(s)
remove_existing (bool) – boolean indicating if an existing comparision model or file(s) should be replaced (default is True)
- setup_mf6(src, dst, mfnamefile='mfsim.nam', extrafiles=None, remove_existing=True)[source]¶
Copy all of the MODFLOW 6 input files from the src directory to the dst directory.
- Parameters
src (src) – directory path with original MODFLOW 6 input files
dst (str) – directory path that original MODFLOW 6 input files will be copied to
mfnamefile (str) – optional MODFLOW 6 simulation name file (default is mfsim.nam)
extrafiles (bool) – boolean indicating if extra files should be included (default is None)
remove_existing (bool) – boolean indicating if existing file in dst should be removed (default is True)
- Returns
mf6inp (list) – list of MODFLOW 6 input files
mf6outp (list) – list of MODFLOW 6 output files
- setup_mf6_comparison(src, dst, remove_existing=True)[source]¶
Setup comparision for MODFLOW 6 simulation.
- Parameters
src (src) – directory path with original MODFLOW 6 input files
dst (str) – directory path that original MODFLOW 6 input files will be copied to
remove_existing (bool) – boolean indicating if existing file in dst should be removed (default is True)
- Returns
action – comparison type
- Return type
str
Module contents¶
Autotest functionality for MODFLOW-based models. Includes functionality for copying existing model and comparision files, comparing results, and tearing down test and comparison models.
pymake.utils package¶
Submodules¶
pymake.utils.download module¶
Utility functions to:
download and unzip software releases from the USGS and other organizations (triangle, MT3DMS).
download the latest MODFLOW-based applications and utilities for MacOS, Linux, and Windows from https://github.com/MODFLOW-USGS/executables
determine the latest version (GitHub tag) of a GitHub repository and a dictionary containing the file name and the link to a asset on contained in a github repository
compress all files in a list, files in a list of directories
- download_and_unzip(url, pth='./', delete_zip=True, verify=True, timeout=30, max_requests=10, chunk_size=2048000, verbose=False)[source]¶
Download and unzip a zip file from a url.
- Parameters
url (str) – url address for the zip file
pth (str) – path where the zip file will be saved (default is the current path)
delete_zip (bool) – boolean indicating if the zip file should be deleted after it is unzipped (default is True)
verify (bool) – boolean indicating if the url request should be verified
timeout (int) – url request time out length (default is 30 seconds)
max_requests (int) – number of url download request attempts (default is 10)
chunk_size (int) – maximum url download request chunk size (default is 2048000 bytes)
verbose (bool) – boolean indicating if output will be printed to the terminal
- get_repo_assets(github_repo=None, version=None, error_return=False, verify=True)[source]¶
Return a dictionary containing the file name and the link to the asset contained in a github repository.
- Parameters
github_repo (str) – Repository name, such as MODFLOW-USGS/modflow6. If github_repo is None set to ‘MODFLOW-USGS/executables’
version (str) – github repository release tag
error_return (bool) – boolean indicating if None will be returned if there are GitHub API issues
verify (bool) – boolean indicating if the url request should be verified
- Returns
result_dict – dictionary of file names and links
- Return type
dict
- getmfexes(pth='.', version=None, platform=None, exes=None, verbose=False, verify=True)[source]¶
Get the latest MODFLOW binary executables from a github site (https://github.com/MODFLOW-USGS/executables) for the specified operating system and put them in the specified path.
- Parameters
pth (str) – Location to put the executables (default is current working directory)
version (str) – Version of the MODFLOW-USGS/executables release to use. If version is None the github repo will be queried for the version number.
platform (str) – Platform that will run the executables. Valid values include mac, linux, win32 and win64. If platform is None, then routine will download the latest asset from the github repository.
exes (str or list of strings) – executable or list of executables to retain
verbose (bool) – boolean indicating if output will be printed to the terminal
verify (bool) – boolean indicating if the url request should be verified
- getmfnightly(pth='.', platform=None, exes=None, verbose=False, verify=True)[source]¶
Get the latest MODFLOW 6 binary nightly-build executables from github (https://github.com/MODFLOW-USGS/modflow6-nightly-build/) for the specified operating system and put them in the specified path.
- Parameters
pth (str) – Location to put the executables (default is current working directory)
platform (str) – Platform that will run the executables. Valid values include mac, linux, win32 and win64. If platform is None, then routine will download the latest asset from the github repository.
exes (str or list of strings) – executable or list of executables to retain
verbose (bool) – boolean indicating if output will be printed to the terminal
verify (bool) – boolean indicating if the url request should be verified
- class pymakeZipFile(file, mode='r', compression=0, allowZip64=True, compresslevel=None)[source]¶
Bases:
ZipFileZipFile file attributes are not being preserved. This class preserves file attributes as described on StackOverflow at https://stackoverflow.com/questions/39296101/python-zipfile-removes-execute-permissions-from-binaries
- static compressall(path, file_pths=None, dir_pths=None, patterns=None)[source]¶
Compress selected files or files in selected directories.
- Parameters
path (str) – output zip file path
file_pths (str or list of str) – file paths to include in the output zip file (default is None)
dir_pths (str or list of str) – directory paths to include in the output zip file (default is None)
patterns (str or list of str) – file patterns to include in the output zip file (default is None)
- Returns
success – boolean indicating if the output zip file was created
- Return type
bool
- extract(member, path=None, pwd=None)[source]¶
- Parameters
member (str) – individual file to extract. If member does not exist, all files are extracted.
path (str) – directory path to extract file in a zip file (default is None, which results in files being extracted in the current directory)
pwd (str) – zip file password (default is None)
- Returns
ret_val – return value indicating status of file extraction
- Return type
int
- extractall(path=None, members=None, pwd=None)[source]¶
Extract all files in the zipfile.
- Parameters
path (str) – directory path to extract files in a zip file (default is None, which results in files being extracted in the current directory)
members (str) – individual files to extract (default is None, which extracts all members)
pwd (str) – zip file password (default is None)
- repo_latest_version(github_repo=None, verify=True)[source]¶
Return a string of the latest version number (tag) contained in a github repository release.
- Parameters
github_repo (str) – Repository name, such as MODFLOW-USGS/modflow6. If github_repo is None set to ‘MODFLOW-USGS/executables’
- Returns
version – string with the latest version/tag number
- Return type
str
- zip_all(path, file_pths=None, dir_pths=None, patterns=None)[source]¶
Compress all files in the user-provided list of file paths and directory paths that match the provided file patterns.
- Parameters
path (str) – path of the zip file that will be created
file_pths (str or list) – file path or list of file paths to be compressed
dir_pths (str or list) – directory path or list of directory paths to search for files that will be compressed
patterns (str or list) – file pattern or list of file patterns s to match to when creating a list of files that will be compressed
pymake.utils.usgsprograms module¶
Utility functions to extract information for a target from the USGS application database. Available functionality includes:
Get a list of available targets
Get data for a specific target
Get a dictionary with the data for all targets
Get the current version of a target
Get a list indicating if single and double precsion versions of the target application should be built
Functions to load, update, and export a USGS-style “code.json” json file containing information in the USGS application database
A table listing the available pymake targets is included below:
- class usgs_program_data[source]¶
Bases:
objectUSGS program database class.
- static export_json(fpth='code.json', prog_data=None, current=False, update=True, write_markdown=False, verbose=False)[source]¶
Export USGS program data as a json file.
- Parameters
fpth (str) – Path for the json file to be created. Default is “code.json”
prog_data (dict) – User-specified program database. If prog_data is None, it will be created from the USGS program database
current (bool) – If False, all USGS program targets are listed. If True, only USGS program targets that are defined as current are listed. Default is False.
update (bool) – If True, existing targets in the user-specified program database with values in the USGS program database. If False, existing targets in the user-specified program database will not be updated. Default is True.
write_markdown (bool) – If True, write markdown file that includes the target name, version, and the last-modified date of the download asset (url). Default is False.
verbose (bool) – boolean for verbose output to terminal
- static get_keys(current=False)[source]¶
Get target keys from the USGS program database.
- Parameters
current (bool) – If False, all USGS program targets are listed. If True, only USGS program targets that are defined as current are listed. Default is False.
- Returns
keys – list of USGS program targets
- Return type
list
- static get_precision(key)[source]¶
Get the dictionary for a specified target.
- Parameters
key (str) – Target USGS program
- Returns
precision – List
- Return type
list
- static get_program_dict()[source]¶
Get the complete USGS program database.
- Returns
program_dict – Dictionary with USGS program attributes for all targets
- Return type
dict
- static get_target(key)[source]¶
Get the dictionary for a specified target.
- Parameters
key (str) – Target USGS program that may have a path and an extension
- Returns
program_dict – Dictionary with USGS program attributes for the specified key
- Return type
dict
- static get_version(key)[source]¶
Get the current version of the specified target.
- Parameters
key (str) – Target USGS program
- Returns
version – current version of the specified target
- Return type
str
- static list_json(fpth='code.json')[source]¶
List an existing code json file.
- Parameters
fpth (str) – Path for the json file to be listed. Default is “code.json”
- static list_targets(current=False)[source]¶
Print a list of the available USGS program targets.
- Parameters
current (bool) – If False, all USGS program targets are listed. If True, only USGS program targets that are defined as current are listed. Default is False.
- static load_json(fpth='code.json')[source]¶
Load an existing code json file. Basic error checking is done to make sure the file contains the correct keys.
- Parameters
fpth (str) – Path for the json file to be created. Default is “code.json”
- Returns
json_dict – Valid USGS program database
- Return type
dict
Module contents¶
Utility functions to 1) download and uncompress software releases containing source code or executables, 2) zip files, 3) query a USGS software and utility program database for information on current software releases.
pymake package¶
Subpackages¶
pymake.cmds package¶
Submodules¶
Download and build USGS MODFLOW and related programs.
This script originates from pymake: https://github.com/modflowpy/pymake It requires Python 3.6 or later, and has no dependencies.
Create pymake code.json file.
This script originates from pymake: https://github.com/modflowpy/pymake It requires Python 3.6 or later, and has no dependencies.
Download and build USGS MODFLOW and related programs.
This script originates from pymake: https://github.com/modflowpy/pymake It requires Python 3.6 or later, and has no dependencies.
Module contents¶
Submodules¶
pymake.config module¶
Module contents¶
pymake is a python package for compiling MODFLOW-based and other Fortran, C,
and C++ programs. The package determines the build order using a directed
acyclic graph and then compiles the source files using GNU compilers
(gcc, g++, gfortran) or Intel compilers
(ifort, icc).
- class Pymake(name='pymake', verbose=None)[source]¶
Bases:
objectPymake class for interacting with pymake functionality. This is essentially a wrapper for all of the pymake functions needed to download and build a target.
- argv_reset_settings(args)[source]¶
Reset settings using command line arguments
- Parameters
args (Namespace object) – reset self.variables using command line arguments
- build(target=None, srcdir=None, modify_exe_name=False)[source]¶
Build the target
- Parameters
target (str) – target name. If target is None self.target is used. (default is None)
srcdir (str) – path to directory with source files. (default is None)
modify_exe_name (bool) – boolean that determines if the target name can be modified to include precision (dbl) and debugging (d) indicators.
- download_setup(target, url=None, download_path='.', verify=True, timeout=30)[source]¶
Setup download
- Parameters
target (str) – target name
url (str) – url of asset
download_path (str) – path where the asset will be saved
verify (bool) – boolean defining ssl verification
timeout (int) – download timeout in seconds (default is 30)
- download_target(target, url=None, download_path='.', verify=True, timeout=30)[source]¶
Setup and download url
- Parameters
target (str) – target name
url (str) – url of asset
download_path (str) – path where the asset will be saved
verify (bool) – boolean defining ssl verification
timeout (int) – download timeout in seconds (default is 30)
- Returns
success – boolean flag indicating download success
- Return type
bool
- download_url()[source]¶
Download files from the url
- Returns
success – boolean flag indicating download success
- Return type
bool
- reset(target)[source]¶
Reset PyMake object variables for a target
- Parameters
target (str) – target name
- set_build_target_bool(target=None)[source]¶
Evaluate if the executable exists and if so and the command line argument –keep is specified then the executable is not built.
- Parameters
target (str) – target name. If target is None self.target will be used. (default is None)
- Returns
build – boolean indicating if the executable should be built
- Return type
bool
- update_target(target, modify_target=False)[source]¶
Update target name with executable extension on Windows and based on pymake settings.
- Parameters
target (str) – target name
modify_target (bool) – boolean indicating if the target name can be modified based on pymake double and debug settings (default is False)
- Returns
target – updated target name
- Return type
str
- build_apps(targets=None, pymake_object=None, download_dir=None, appdir=None, verbose=None, release_precision=True, meson=False, mesondir='.', clean=True)[source]¶
Build all of the current targets or a subset of targets.
- Parameters
targets (str or list of str) – targets to build. If targets is None, all current targets will be build. Default is None
pymake_object (Pymake()) – Pymake object created outside of build_apps
download_dir (str) – download directory path
appdir (str) – target path
release_precision (bool) – boolean indicating if only the release precision version should be build. If release_precision is False, then the release precision version will be compiled along with a double precision version of the program for programs where the standard_switch and double_switch in usgsprograms.txt is True. default is True.
meson (bool) – boolean indicating that the executable should be built using the meson build system. (default is False)
mesondir (str) – Main meson.build file path
clean (bool) – boolean determining of final download should be removed
- Returns
returncode – integer value indicating successful completion (0) or failure (>0)
- Return type
int
- compare(namefile1, namefile2, precision='auto', max_cumpd=0.01, max_incpd=0.01, htol=0.001, outfile1=None, outfile2=None, files1=None, files2=None)[source]¶
Compare the budget and head results for two MODFLOW-based model simulations.
- Parameters
namefile1 (str) – namefile path for base model
namefile2 (str) – namefile path for comparison model
precision (str) – precision for binary head file (“auto”, “single”, or “double”) default is “auto”
max_cumpd (float) – maximum percent discrepancy allowed for cumulative budget terms (default is 0.01)
max_incpd (float) – maximum percent discrepancy allowed for incremental budget terms (default is 0.01)
htol (float) – maximum allowed head difference (default is 0.001)
outfile1 (str) – budget comparison output file name. If outfile1 is None, no budget comparison output is saved. (default is None)
outfile2 (str) – head comparison output file name. If outfile2 is None, no head comparison output is saved. (default is None)
files1 (str) – base model output file. If files1 is not None, results will be extracted from files1 and namefile1 will not be used. (default is None)
files2 (str) – comparison model output file. If files2 is not None, results will be extracted from files2 and namefile2 will not be used. (default is None)
- Returns
success – boolean indicating if the budget and head differences are less than max_cumpd, max_incpd, and htol.
- Return type
bool
- compare_budget(namefile1, namefile2, max_cumpd=0.01, max_incpd=0.01, outfile=None, files1=None, files2=None)[source]¶
Compare the budget results from two simulations.
- Parameters
namefile1 (str) – namefile path for base model
namefile2 (str) – namefile path for comparison model
max_cumpd (float) – maximum percent discrepancy allowed for cumulative budget terms (default is 0.01)
max_incpd (float) – maximum percent discrepancy allowed for incremental budget terms (default is 0.01)
outfile (str) – budget comparison output file name. If outfile is None, no comparison output is saved. (default is None)
files1 (str) – base model output file. If files1 is not None, results will be extracted from files1 and namefile1 will not be used. (default is None)
files2 (str) – comparison model output file. If files2 is not None, results will be extracted from files2 and namefile2 will not be used. (default is None)
- Returns
success – boolean indicating if the difference between budgets are less than max_cumpd and max_incpd
- Return type
bool
- compare_concs(namefile1, namefile2, precision='auto', ctol=0.001, outfile=None, files1=None, files2=None, difftol=False, verbose=False)[source]¶
Compare the mt3dms and mt3dusgs concentration results from two simulations.
- Parameters
namefile1 (str) – namefile path for base model
namefile2 (str) – namefile path for comparison model
precision (str) – precision for binary head file (“auto”, “single”, or “double”) default is “auto”
ctol (float) – maximum allowed concentration difference (default is 0.001)
outfile (str) – concentration comparison output file name. If outfile is None, no comparison output is saved. (default is None)
files1 (str) – base model output file. If files1 is not None, results will be extracted from files1 and namefile1 will not be used. (default is None)
files2 (str) – comparison model output file. If files2 is not None, results will be extracted from files2 and namefile2 will not be used. (default is None)
difftol (bool) – boolean determining if the absolute value of the concentration difference greater than ctol should be evaluated (default is False)
verbose (bool) – boolean indicating if verbose output should be written to the terminal (default is False)
- Returns
success – boolean indicating if the concentration differences are less than ctol.
- Return type
bool
- compare_heads(namefile1, namefile2, precision='auto', text='head', text2=None, htol=0.001, outfile=None, files1=None, files2=None, difftol=False, verbose=False, exfile=None, exarr=None, maxerr=None)[source]¶
Compare the head results from two simulations.
- Parameters
namefile1 (str) – namefile path for base model
namefile2 (str) – namefile path for comparison model
precision (str) – precision for binary head file (“auto”, “single”, or “double”) default is “auto”
htol (float) – maximum allowed head difference (default is 0.001)
outfile (str) – head comparison output file name. If outfile is None, no comparison output is saved. (default is None)
files1 (str) – base model output file. If files1 is not None, results will be extracted from files1 and namefile1 will not be used. (default is None)
files2 (str) – comparison model output file. If files2 is not None, results will be extracted from files2 and namefile2 will not be used. (default is None)
difftol (bool) – boolean determining if the absolute value of the head difference greater than htol should be evaluated (default is False)
verbose (bool) – boolean indicating if verbose output should be written to the terminal (default is False)
exfile (str) – path to a file with exclusion array data. Head differences will not be evaluated where exclusion array values are greater than zero. (default is None)
exarr (numpy.ndarry) – exclusion array. Head differences will not be evaluated where exclusion array values are greater than zero. (default is None).
maxerr (int) – maximum number of head difference greater than htol that should be reported. If maxerr is None, all head difference greater than htol will be reported. (default is None)
- Returns
success – boolean indicating if the head differences are less than htol.
- Return type
bool
- compare_stages(namefile1=None, namefile2=None, files1=None, files2=None, htol=0.001, outfile=None, difftol=False, verbose=False)[source]¶
Compare SWR process stage results from two simulations.
- Parameters
namefile1 (str) – namefile path for base model
namefile2 (str) – namefile path for comparison model
precision (str) – precision for binary head file (“auto”, “single”, or “double”) default is “auto”
htol (float) – maximum allowed stage difference (default is 0.001)
outfile (str) – head comparison output file name. If outfile is None, no comparison output is saved. (default is None)
files1 (str) – base model output file. If files1 is not None, results will be extracted from files1 and namefile1 will not be used. (default is None)
files2 (str) – comparison model output file. If files2 is not None, results will be extracted from files2 and namefile2 will not be used. (default is None)
difftol (bool) – boolean determining if the absolute value of the stage difference greater than htol should be evaluated (default is False)
verbose (bool) – boolean indicating if verbose output should be written to the terminal (default is False)
- Returns
success – boolean indicating if the stage differences are less than htol.
- Return type
bool
- compare_swrbudget(namefile1, namefile2, max_cumpd=0.01, max_incpd=0.01, outfile=None, files1=None, files2=None)[source]¶
Compare the SWR budget results from two simulations.
- Parameters
namefile1 (str) – namefile path for base model
namefile2 (str) – namefile path for comparison model
max_cumpd (float) – maximum percent discrepancy allowed for cumulative budget terms (default is 0.01)
max_incpd (float) – maximum percent discrepancy allowed for incremental budget terms (default is 0.01)
outfile (str) – budget comparison output file name. If outfile is None, no comparison output is saved. (default is None)
files1 (str) – base model output file. If files1 is not None, results will be extracted from files1 and namefile1 will not be used. (default is None)
files2 (str) – comparison model output file. If files2 is not None, results will be extracted from files2 and namefile2 will not be used. (default is None)
- Returns
success – boolean indicating if the difference between budgets are less than max_cumpd and max_incpd
- Return type
bool
- download_and_unzip(url, pth='./', delete_zip=True, verify=True, timeout=30, max_requests=10, chunk_size=2048000, verbose=False)[source]¶
Download and unzip a zip file from a url.
- Parameters
url (str) – url address for the zip file
pth (str) – path where the zip file will be saved (default is the current path)
delete_zip (bool) – boolean indicating if the zip file should be deleted after it is unzipped (default is True)
verify (bool) – boolean indicating if the url request should be verified
timeout (int) – url request time out length (default is 30 seconds)
max_requests (int) – number of url download request attempts (default is 10)
chunk_size (int) – maximum url download request chunk size (default is 2048000 bytes)
verbose (bool) – boolean indicating if output will be printed to the terminal
- get_entries_from_namefile(namefile, ftype=None, unit=None, extension=None)[source]¶
Get entries from a namefile. Can select using FTYPE, UNIT, or file extension.
- Parameters
namefile (str) – path to a MODFLOW-based model name file
ftype (str) – package type
unit (int) – file unit number
extension (str) – file extension
- Returns
entries – list of tuples containing FTYPE, UNIT, FNAME, STATUS for each namefile entry that meets a user-specified value.
- Return type
list of tuples
- get_input_files(namefile)[source]¶
Return a list of all the input files in this model.
- Parameters
namefile (str) – path to a MODFLOW-based model name file
- Returns
filelist – list of MODFLOW-based model input files
- Return type
list
- get_mf6_blockdata(f, blockstr)[source]¶
Return list with all non comments between start and end of block specified by blockstr.
- Parameters
f (file object) – open file object
blockstr (str) – name of block to search
- Returns
data – list of data in specified block
- Return type
list
- get_mf6_comparison(src)[source]¶
Determine comparison type for MODFLOW 6 simulation.
- Parameters
src (str) – directory path to search for comparison types
- Returns
action – comparison type
- Return type
str
- get_mf6_files(mfnamefile)[source]¶
Return a list of all the MODFLOW 6 input and output files in this model.
- Parameters
mfnamefile (str) – path to the MODFLOW 6 simulation name file
- Returns
filelist (list) – list of MODFLOW 6 input files in a simulation
outplist (list) – list of MODFLOW 6 output files in a simulation
- get_mf6_ftypes(namefile, ftypekeys)[source]¶
Return a list of FTYPES that are in the name file and in ftypekeys.
- Parameters
namefile (str) – path to a MODFLOW 6 name file
ftypekeys (list) – list of desired FTYPEs
- Returns
ftypes – list of FTYPES that match ftypekeys in namefile
- Return type
list
- get_mf6_mshape(disfile)[source]¶
Return the shape of the MODFLOW 6 model.
- Parameters
disfile (str) – path to a MODFLOW 6 discretization file
- Returns
mshape – tuple with the shape of the MODFLOW 6 model.
- Return type
tuple
- get_mf6_nper(tdisfile)[source]¶
Return the number of stress periods in the MODFLOW 6 model.
- Parameters
tdisfile (str) – path to the TDIS file
- Returns
nper – number of stress periods in the simulation
- Return type
int
- get_namefiles(pth, exclude=None)[source]¶
Search through a path (pth) for all .nam files.
- Parameters
pth (str) – path to model files
exclude (str or lst) – File or list of files to exclude from the search (default is None)
- Returns
namefiles – List of namefiles with paths
- Return type
lst
- get_repo_assets(github_repo=None, version=None, error_return=False, verify=True)[source]¶
Return a dictionary containing the file name and the link to the asset contained in a github repository.
- Parameters
github_repo (str) – Repository name, such as MODFLOW-USGS/modflow6. If github_repo is None set to ‘MODFLOW-USGS/executables’
version (str) – github repository release tag
error_return (bool) – boolean indicating if None will be returned if there are GitHub API issues
verify (bool) – boolean indicating if the url request should be verified
- Returns
result_dict – dictionary of file names and links
- Return type
dict
- get_sim_name(namefiles, rootpth=None)[source]¶
Get simulation name.
- Parameters
namefiles (str or list of strings) – path(s) to MODFLOW-based model name files
rootpth (str) – optional root directory path (default is None)
- Returns
simname – list of namefiles without the file extension
- Return type
list
- getmfexes(pth='.', version=None, platform=None, exes=None, verbose=False, verify=True)[source]¶
Get the latest MODFLOW binary executables from a github site (https://github.com/MODFLOW-USGS/executables) for the specified operating system and put them in the specified path.
- Parameters
pth (str) – Location to put the executables (default is current working directory)
version (str) – Version of the MODFLOW-USGS/executables release to use. If version is None the github repo will be queried for the version number.
platform (str) – Platform that will run the executables. Valid values include mac, linux, win32 and win64. If platform is None, then routine will download the latest asset from the github repository.
exes (str or list of strings) – executable or list of executables to retain
verbose (bool) – boolean indicating if output will be printed to the terminal
verify (bool) – boolean indicating if the url request should be verified
- main(srcdir=None, target=None, fc='gfortran', cc='gcc', makeclean=True, expedite=False, dryrun=False, double=False, debug=False, include_subdirs=False, fflags=None, cflags=None, syslibs=None, arch='intel64', makefile=False, makefiledir='.', srcdir2=None, extrafiles=None, excludefiles=None, sharedobject=False, appdir=None, verbose=False, inplace=False, networkx=False, meson=False, mesondir='.')[source]¶
Main pymake function.
- Parameters
srcdir (str) – path for directory containing source files
target (str) – executable name or path for executable to create
fc (str) – fortran compiler
cc (str) – c or cpp compiler
makeclean (bool) – boolean indicating if intermediate files should be cleaned up after successful build
expedite (bool) – boolean indicating if only out of date source files will be compiled. Clean must not have been used on previous build.
dryrun (bool) – boolean indicating if source files should be compiled. Files will be deleted, if makeclean is True.
double (bool) – boolean indicating a compiler switch will be used to create an executable with double precision real variables.
debug (bool) – boolean indicating is a debug executable will be built
include_subdirs (bool) – boolean indicating source files in srcdir subdirectories should be included in the build
fflags (list) – user provided list of fortran compiler flags
cflags (list) – user provided list of c or cpp compiler flags
syslibs (list) – user provided syslibs
arch (str) – Architecture to use for Intel Compilers on Windows (default is intel64)
makefile (bool) – boolean indicating if a GNU make makefile should be created
makefiledir (str) – GNU make makefile path
srcdir2 (str) – additional directory with common source files.
extrafiles (str) – path for extrafiles file that contains paths to additional source files to include
excludefiles (str) – path for excludefiles file that contains filename of source files to exclude from the build
sharedobject (bool) – boolean indicating a shared object will be built
appdir (str) – path for executable
verbose (bool) – boolean indicating if output will be printed to the terminal
inplace (bool) – boolean indicating that the source files in srcdir, srcdir2, and defined in extrafiles will be used directly. If inplace is False, source files will be copied to a directory named srcdir_temp. (default is False)
networkx (bool) – boolean indicating that the NetworkX python package will be used to create the Directed Acyclic Graph (DAG) used to determine the order source files are compiled in. The NetworkX package tends to result in a unique DAG more often than the standard algorithm used in pymake. (default is False)
meson (bool) – boolean indicating that the executable should be built using the meson build system. (default is False)
mesondir (str) – Main meson.build file path
- Returns
returncode – return code
- Return type
int
- make_plots(srcdir, outdir, include_subdir=False, level=3, extension='.png', verbose=False, networkx=False)[source]¶
Create plots of module dependencies.
- Parameters
srcdir (str) – path for source files
outdir (str) – path for output images
include_subdir (bool) – boolean indicating is subdirectories in the source file directory should be included
level (int) – dependency level (1 is the minimum)
extension (str) – output extension (default is .png)
verbose (bool) – boolean indicating if output will be printed to the terminal
networkx (bool) – boolean indicating that the NetworkX python package will be used to create the Directed Acyclic Graph (DAG) used to determine the order source files are compiled in. The NetworkX package tends to result in a unique DAG more often than the standard algorithm used in pymake. (default is False)
- parser(examples=None)[source]¶
Construct the parser and return argument values.
- Parameters
examples (str) –
- Returns
args – Namespace with command line arguments
- Return type
Namespace object
- repo_latest_version(github_repo=None, verify=True)[source]¶
Return a string of the latest version number (tag) contained in a github repository release.
- Parameters
github_repo (str) – Repository name, such as MODFLOW-USGS/modflow6. If github_repo is None set to ‘MODFLOW-USGS/executables’
- Returns
version – string with the latest version/tag number
- Return type
str
- setup(namefile, dst, remove_existing=True, extrafiles=None)[source]¶
Setup MODFLOW-based model files for autotests.
- Parameters
namefile (str) – MODFLOW-based model name file.
dst (str) – destination path for comparison model or file(s)
remove_existing (bool) – boolean indicating if an existing comparision model or file(s) should be replaced (default is True)
extrafiles (str or list of str) – list of extra files to include in the comparision
- setup_comparison(namefile, dst, remove_existing=True)[source]¶
Setup a comparison model or comparision file(s) for a MODFLOW-based model.
- Parameters
namefile (str) – MODFLOW-based model name file.
dst (str) – destination path for comparison model or file(s)
remove_existing (bool) – boolean indicating if an existing comparision model or file(s) should be replaced (default is True)
- setup_mf6(src, dst, mfnamefile='mfsim.nam', extrafiles=None, remove_existing=True)[source]¶
Copy all of the MODFLOW 6 input files from the src directory to the dst directory.
- Parameters
src (src) – directory path with original MODFLOW 6 input files
dst (str) – directory path that original MODFLOW 6 input files will be copied to
mfnamefile (str) – optional MODFLOW 6 simulation name file (default is mfsim.nam)
extrafiles (bool) – boolean indicating if extra files should be included (default is None)
remove_existing (bool) – boolean indicating if existing file in dst should be removed (default is True)
- Returns
mf6inp (list) – list of MODFLOW 6 input files
mf6outp (list) – list of MODFLOW 6 output files
- setup_mf6_comparison(src, dst, remove_existing=True)[source]¶
Setup comparision for MODFLOW 6 simulation.
- Parameters
src (src) – directory path with original MODFLOW 6 input files
dst (str) – directory path that original MODFLOW 6 input files will be copied to
remove_existing (bool) – boolean indicating if existing file in dst should be removed (default is True)
- Returns
action – comparison type
- Return type
str
- teardown(src)[source]¶
Teardown a autotest directory.
- Parameters
src (str) – autotest directory to teardown
- to_pydot(dag, filename='mygraph.png')[source]¶
Create a png file of a Directed Acyclic Graph
- Parameters
dag (object) – directed acyclic graph
filename (str) – path of the graph png
- class usgs_program_data[source]¶
Bases:
objectUSGS program database class.
- static export_json(fpth='code.json', prog_data=None, current=False, update=True, write_markdown=False, verbose=False)[source]¶
Export USGS program data as a json file.
- Parameters
fpth (str) – Path for the json file to be created. Default is “code.json”
prog_data (dict) – User-specified program database. If prog_data is None, it will be created from the USGS program database
current (bool) – If False, all USGS program targets are listed. If True, only USGS program targets that are defined as current are listed. Default is False.
update (bool) – If True, existing targets in the user-specified program database with values in the USGS program database. If False, existing targets in the user-specified program database will not be updated. Default is True.
write_markdown (bool) – If True, write markdown file that includes the target name, version, and the last-modified date of the download asset (url). Default is False.
verbose (bool) – boolean for verbose output to terminal
- static get_keys(current=False)[source]¶
Get target keys from the USGS program database.
- Parameters
current (bool) – If False, all USGS program targets are listed. If True, only USGS program targets that are defined as current are listed. Default is False.
- Returns
keys – list of USGS program targets
- Return type
list
- static get_precision(key)[source]¶
Get the dictionary for a specified target.
- Parameters
key (str) – Target USGS program
- Returns
precision – List
- Return type
list
- static get_program_dict()[source]¶
Get the complete USGS program database.
- Returns
program_dict – Dictionary with USGS program attributes for all targets
- Return type
dict
- static get_target(key)[source]¶
Get the dictionary for a specified target.
- Parameters
key (str) – Target USGS program that may have a path and an extension
- Returns
program_dict – Dictionary with USGS program attributes for the specified key
- Return type
dict
- static get_version(key)[source]¶
Get the current version of the specified target.
- Parameters
key (str) – Target USGS program
- Returns
version – current version of the specified target
- Return type
str
- static list_json(fpth='code.json')[source]¶
List an existing code json file.
- Parameters
fpth (str) – Path for the json file to be listed. Default is “code.json”
- static list_targets(current=False)[source]¶
Print a list of the available USGS program targets.
- Parameters
current (bool) – If False, all USGS program targets are listed. If True, only USGS program targets that are defined as current are listed. Default is False.
- static load_json(fpth='code.json')[source]¶
Load an existing code json file. Basic error checking is done to make sure the file contains the correct keys.
- Parameters
fpth (str) – Path for the json file to be created. Default is “code.json”
- Returns
json_dict – Valid USGS program database
- Return type
dict
- zip_all(path, file_pths=None, dir_pths=None, patterns=None)[source]¶
Compress all files in the user-provided list of file paths and directory paths that match the provided file patterns.
- Parameters
path (str) – path of the zip file that will be created
file_pths (str or list) – file path or list of file paths to be compressed
dir_pths (str or list) – directory path or list of directory paths to search for files that will be compressed
patterns (str or list) – file pattern or list of file patterns s to match to when creating a list of files that will be compressed