Basic usage¶
Changing the current working directory:
from pyworkdir import WorkDir
with WorkDir("some_directory"):
# everything in this context is run
# in the specified directory
pass
Directories are Customizable Classes¶
WorkDir classes can be be customized by adding a file workdir.py to the directory. All variables, functions, or classes defined in this file will be added as attributes of the WorkDir instances.
For instance, consider the following workdir.py file:
# -- workdir.py --
def data_file(workdir, filename="data.csv"):
return workdir/filename
The function can now be accessed from other code as follows:
from pyworkdir import WorkDir
with WorkDir() as wd:
print(wd.data_file())
Note that the parameter workdir behaves like the self argument of the method. If workdir is not an argument of the function, the function behaves like a static method.
By default, the WorkDir instance also recursively inherits attributes defined in its parent directory’s workdir.py files. Therefore, subdirectories behave like subclasses.
Directories have a Command Line Interface¶
Custom functions of the WorkDir are directly accessible from a terminal via the command workdir. Before being called from the command line, all function parameters (except the reserved keywords workdir and here) have to be declared as Click options:
# -- workdir.py --
import click
num_apples = 2
@click.option("-c", type=int, default=12, help="A number (default:12)")
@click.option("-s","--somebody", type=str, help="A name")
def hello(count, somebody, workdir):
"""This function says hello."""
workdir.num_apples += 1
print(
f"{count} times Hello! to {somebody}: "
f"we have {workdir.num_apples} apples."
)
Calling the function from the command line looks like this:
foo@bar:~$ workdir hello --help
Usage: workdir hello [OPTIONS]
This function says hello.
Options:
-c, --count INTEGER A number (default:12)
-s, --somebody TEXT A name
--help Show this message and exit.
foo@bar:~$ workdir hello -s "you"
12 times Hello! to you: we have 3 apples.
Writing workdir.py files like this makes it easy to define local functions that can be called both from inside python and from a terminal. For the latter, the workdir.py behaves similar to a Makefile.
To suppress generation of the command line interface for a function, pyworkdir provides a no_cli decorator:
# -- workdir.py --
from pyworkdir import no_cli
@no_cli
def a_function_without_command_line_interface():
pass
Changing Environment Variables¶
Temporary changes of the environment:
from pyworkdir import WorkDir
with WorkDir(environment={"MY_ENVIRONMENT_VARIABLE":"1"}):
# in this context the environment variable is set
pass
# outside the context, it is not set any longer
Yaml Files¶
Environment variables and simple attributes can also be set through yml files. The templates {{ workdir }} and {{ here }} are available and will be replaced by the working directory instance and the directory that contains the yml file, respectively:
# -- workdir.yml --
environment:
VAR_ONE: "a"
attributes:
my_number: 1
my_list:
- 1
- 2
- 3
my_tmpdir: {{ here/"tmpdir" }}
my_local_tmpfile: {{ workdir/"file.tmp" }}
commands:
echo: echo Hello // print Hello to the command line
The commands are shortcuts for terminal commands that can be called from python and from the command line. Everything after // is used as a documentation string for the command line interface. The attributes and environment variables get added to the WorkDir:
import os
with WorkDir() as wd:
print(wd.my_number + 5, wd.my_tmpdir , wd.my_local_tmpfile)
for el in wd.my_list:
print(el)
print(os.environ["VAR_ONE"])
Note that environment variables passed to the constructor have preference over those in a yml file.
Logging¶
A logger is available:
from pyworkdir import WorkDir
import logging
wd = WorkDir()
wd.log("a INFO-level message")
wd.log("a DEBUG-level message", logging.DEBUG)
By default, INFO-level and higher is printed to the console. DEBUG-level output is only printed to a file workdir.log.
API¶
workdir¶
Python working directories.
-
class
pyworkdir.workdir.
WorkDir
(directory='.', mkdir=True, python_files=['workdir.py'], yml_files=['workdir.yml'], python_files_recursion=-1, yml_files_recursion=-1, environment={}, logger=None, logfile='workdir.log', loglevel_console=20, loglevel_file=10) Bases:
object
Working directory class.
- Parameters
directory (str, Optional, default: ".") – The directory name
mkdir (bool, Optional, default: True) – Whether to create the directory if it does not exist
python_files (list of string, Optional, default: ["workdir.py"]) – A list of python files. All variables, functions, and classes defined in these files are added as members to customize the WorkDir.
yml_files (list of string, Optional, default: ["workdir.yml"]) – A list of configuration files to read a configuration from.
python_files_recursion (int, Optional, default: -1) – Recursion level for loading python files from parent directories. 0 means only this directory, 1 means this directory and its parent directory, etc. If -1, recurse until root.
yml_files_recursion (int, Optional, default: -1) – Recursion level for yml files.
environment (dict, Optional, default: dict()) – A dictionary. Keys (names of environment variables) and values (values of environment variables) have to be strings. Environment variables are temporarily set to these values within a context (a with WorkDir() … block) and set to their original values outside the context.
logger (logging.Logger or None, Optional, default: None) – A logger instance. If None, use a default logger. If a custom logger is specified, the other arguments that concern the logger are not recognized.
logfile (str, Optional, default: "workdir.log") – The logfile to write output to.
loglevel_console (int, Optional, default: logging.INFO) – The level of logging to the console.
loglevel_file (int, Optional, default: logging.DEBUG) – The level of logging to the logfile.
-
path
¶ Absolute path of this working directory
- Type
pathlib.Path
-
scope_path
¶ The path of the surrounding scope (when used as a context manager)
- Type
pathlib.Path
-
environment
¶ A dictionary of environment variables to be set in the context
- Type
dict
-
scope_environment
¶ A dictionary to keep track of the environment of the scope
- Type
dict
-
custom_attributes
¶ A dictionary that lists custom attributes of this working directory. The values of the dictionary are the source files which contain the definition of each attribute.
- Type
dict
-
python_files
¶ A list of python filenames that the workdir instance may read its custom attributes from. Files do not need to exist.
- Type
list of str
-
yml_files
¶ A list of yml filenames that the workdir instance may read its custom attributes from. Files do not need to exist.
- Type
list of str
-
logger
¶ A logger instance
- Type
logging.Logger or None
-
logfile
¶ Filename of the log file
- Type
str
-
loglevel_console
¶ An integer between 0 (logging.NOT_SET) and 50 (logging.CRITICAL) for level of printing to the console
- Type
int
-
loglevel_file
¶ An integer between 0 (logging.NOT_SET) and 50 (logging.CRITICAL) for level of printing to the file
- Type
int
-
commands
¶ A dictionary of terminal commands.
- Type
dict
Notes
Get the absolute path of a file in this working directory
>>> with WorkDir("some_path") as wd: >>> absolute_path = wd / "some_file.txt"
Get the number of files and subdirectories:
>>> len(wd)
Iterate over all files in this working directory:
>>> for f in wd.files(): >>> pass
Examples
Basic usage:
>>> with WorkDir("some_path"): >>> # everything in this context will >>> # run in the specified directory >>> pass
Customizing the working directory:
To add or change attributes of the WorkDir, create a file “workdir.py” in the directory. All functions, classes, and variables defined in “workdir.py” will be added as attributes to the WorkDir.
>>> # -- workdir.py -- >>> def custom_sum_function(a, b): >>> return a + b
>>> # -- main.py -- >>> wd = WorkDir(".") >>> result = wd.custom_sum_function(a,b)
By default, these attributes get added recursively from parent directories as well, where more specific settings (further down in the directory tree) override more general ones. This mimics a kind of inheritance, where subdirectories inherit attributes from their parents.
When defining functions in the workdir.py file, some argument names have special meaning: - The argument name workdir refers to the working directory instance.
It represents the self argument of the method.
The argument name here refers to the absolute path of the directory that contains the workdir.py file.
Environment variables can be changed inside a context as follows.
>>> import os >>> with WorkDir(environment={"VAR_ONE": "ONE", "VAR_TWO": "TWO"}): >>> print(os.environ["VAR_ONE"]) >>> assert "VAR_ONE" not in os.environ
Environment variables and simple attributes can also be set through yml files. The templates {{ workdir }} and {{ here }} are available and will be replaced by the working directory instance and the directory that contains the yml file, respectively.
>>> # -- workdir.yml -- environment: VAR_ONE: "a" attributes: my_number: 1 my_list: - 1 - 2 - 3 my_tmpdir: {{ here/"tmpdir" }} my_local_tmpfile: {{ workdir/"file.tmp" }}
>>> with WorkDir() as wd: >>> print(wd.my_number + 5, wd.my_tmpdir , wd.my_local_tmpfile) >>> for el in wd.my_list: >>> print(el) >>> print(os.environ["VAR_ONE"])
Note that environment variables passed to the constructor have preference over those in a yml file.
A logging instance is available; the default output file is workdir.log:
>>> wd = WorkDir() >>> wd.log("my message") >>> import logging >>> wd.log("debug info", level=logging.DEBUG)
-
add_members_from_pyfile
(pyfile) Initialize members of this WorkDir from a python file.
The following attributes are not added as members of the WorkDir:
imported modules
built-ins and private objects, i.e. if the name starts with an underscore
objects that are imported from other modules using from … import …
The only exception to 3. is if the imported function has a command-line interface, i.e. @click.option-decorated functions added to the workdir so that they can be called from the command line.
- Parameters
pyfile (path-like object) – Absolute path of a python file.
Notes
The function arguments workdir and here of imported functions are replaced by the WorkDir instance and the directory containing the pyfile, respectively.
-
add_members_from_yml_file
(yml_file) Initialize members and environment variables from a yml file.
-
files
(abs=False) Iterator over files in this work dir.
- Parameters
abs (bool, Optional, default=False) – Yield absolute filenames
- Yields
file (str) – Filenames in this directory
Examples
>>> with WorkDir("some_directory") as wd: >>> for file in wd.files(): >>> print(file)
-
log
(message, level=20) Write logging output to the console and/or a log file.
- Parameters
message (str) –
level (int, Optional, default: logging.DEBUG) –
util¶
Utilities for workdir
-
exception
pyworkdir.util.
WorkDirException
Bases:
Exception
General exception class for pyworkdir module.
-
pyworkdir.util.
forge_method
(instance, func, replace_args={}, name=None, add=True) Forge a method and add it to an instance.
- Parameters
instance (class instance) – The instance to which the function should be added as a method
func (function) – The function to be added to the instance
replace_args (dict, Optional, default = dict()) – Any arguments that are replaced by default values in the spirit of functools.partial
name (str, Optional, default=None) – The function’s name; if None, infer from function.__name__
add (bool, Optional, default=True) – If False, do not add the function but return it instead.
Notes
This function takes care of option-decorated functions. They retain their __click_params__ field; also all replace_args get added as hidden options so that they are not visible on the command line interface.
-
pyworkdir.util.
import_from_file
(filename) Import a python module from a file by path.
- Parameters
filename (str or path-like) – The file to be imported
- Returns
pymod – The imported module
- Return type
python module
-
pyworkdir.util.
recursively_get_filenames
(path, filenames, recursion_depth, current_recursion_level=0) Get all filenames (python/yaml) that attributes should be read from.
- Parameters
path (str or path-like object) – the current directory
filenames (str) – The base filenames.
recursion_depth (int) – The maximum recursion depth (0 = only current directory, 1 = current and parents). -1 means recurse until root.
current_recursion_level (int, Optional, default = 0) – Current recursion level of the function.
- Returns
filenames – A list of filenames, where the ones further up front in the list are further up in the directory tree. The files do not need to exist.
- Return type
list
main¶
Command line interface
-
pyworkdir.main.
bash_function
(bash_command) Bash command as a python function
- Parameters
bash_command (str) –
- Returns
function – A python function that runs the bash command.
- Return type
callable
-
pyworkdir.main.
entrypoint
() Entrypoint for the workdir command.
-
pyworkdir.main.
forge_command_line_interface
(*args, **kwargs) Forge the click.Group that holds all commands defined in workdir.py All arguments are forwarded to the constructor of WorkDir.
- Returns
- Return type
A click.Group that defines one command for every custom function in the WorkDir.
-
pyworkdir.main.
no_cli
(function) Function decorator to suppress generation of a command-line interface for this function.
Examples
>>> # in workdir.py >>> from pyworkdir import no_cli >>> >>> @no_cli >>> def function_without_command_line_interace() >>> pass
-
pyworkdir.main.
show
(workdir, variables=False, functions=False, sources=False, environment=False, commands=False, out=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>) Print the working directory in yaml format.