pocket_coffea.law_tasks.configuration package

class pocket_coffea.law_tasks.configuration.general.baseconfig(*args, **kwargs)#

Bases: Config

config class that holds general parameters

cfg#

Parameter whose value is a str, and a base class for other parameter types.

Parameters are objects set on the Task class level to make it possible to parameterize tasks. For instance:

class MyTask(luigi.Task):
    foo = luigi.Parameter()

class RequiringTask(luigi.Task):
    def requires(self):
        return MyTask(foo="hello")

    def run(self):
        print(self.requires().foo)  # prints "hello"

This makes it possible to instantiate multiple tasks, eg MyTask(foo='bar') and MyTask(foo='baz'). The task will then have the foo attribute set appropriately.

When a task is instantiated, it will first use any argument as the value of the parameter, eg. if you instantiate a = TaskA(x=44) then a.x == 44. When the value is not provided, the value will be resolved in this order of falling priority:

Parameter objects may be reused, but you must then set the positional=False flag.

class pocket_coffea.law_tasks.configuration.general.datacardconfig(*args, **kwargs)#

Bases: Config

category#

Parameter whose value is a str, and a base class for other parameter types.

Parameters are objects set on the Task class level to make it possible to parameterize tasks. For instance:

class MyTask(luigi.Task):
    foo = luigi.Parameter()

class RequiringTask(luigi.Task):
    def requires(self):
        return MyTask(foo="hello")

    def run(self):
        print(self.requires().foo)  # prints "hello"

This makes it possible to instantiate multiple tasks, eg MyTask(foo='bar') and MyTask(foo='baz'). The task will then have the foo attribute set appropriately.

When a task is instantiated, it will first use any argument as the value of the parameter, eg. if you instantiate a = TaskA(x=44) then a.x == 44. When the value is not provided, the value will be resolved in this order of falling priority:

Parameter objects may be reused, but you must then set the positional=False flag.

datacard_name#

Parameter whose value is a str, and a base class for other parameter types.

Parameters are objects set on the Task class level to make it possible to parameterize tasks. For instance:

class MyTask(luigi.Task):
    foo = luigi.Parameter()

class RequiringTask(luigi.Task):
    def requires(self):
        return MyTask(foo="hello")

    def run(self):
        print(self.requires().foo)  # prints "hello"

This makes it possible to instantiate multiple tasks, eg MyTask(foo='bar') and MyTask(foo='baz'). The task will then have the foo attribute set appropriately.

When a task is instantiated, it will first use any argument as the value of the parameter, eg. if you instantiate a = TaskA(x=44) then a.x == 44. When the value is not provided, the value will be resolved in this order of falling priority:

Parameter objects may be reused, but you must then set the positional=False flag.

shapes_name#

Parameter whose value is a str, and a base class for other parameter types.

Parameters are objects set on the Task class level to make it possible to parameterize tasks. For instance:

class MyTask(luigi.Task):
    foo = luigi.Parameter()

class RequiringTask(luigi.Task):
    def requires(self):
        return MyTask(foo="hello")

    def run(self):
        print(self.requires().foo)  # prints "hello"

This makes it possible to instantiate multiple tasks, eg MyTask(foo='bar') and MyTask(foo='baz'). The task will then have the foo attribute set appropriately.

When a task is instantiated, it will first use any argument as the value of the parameter, eg. if you instantiate a = TaskA(x=44) then a.x == 44. When the value is not provided, the value will be resolved in this order of falling priority:

Parameter objects may be reused, but you must then set the positional=False flag.

stat_config#

Parameter whose value is a str, and a base class for other parameter types.

Parameters are objects set on the Task class level to make it possible to parameterize tasks. For instance:

class MyTask(luigi.Task):
    foo = luigi.Parameter()

class RequiringTask(luigi.Task):
    def requires(self):
        return MyTask(foo="hello")

    def run(self):
        print(self.requires().foo)  # prints "hello"

This makes it possible to instantiate multiple tasks, eg MyTask(foo='bar') and MyTask(foo='baz'). The task will then have the foo attribute set appropriately.

When a task is instantiated, it will first use any argument as the value of the parameter, eg. if you instantiate a = TaskA(x=44) then a.x == 44. When the value is not provided, the value will be resolved in this order of falling priority:

Parameter objects may be reused, but you must then set the positional=False flag.

variable#

Parameter whose value is a str, and a base class for other parameter types.

Parameters are objects set on the Task class level to make it possible to parameterize tasks. For instance:

class MyTask(luigi.Task):
    foo = luigi.Parameter()

class RequiringTask(luigi.Task):
    def requires(self):
        return MyTask(foo="hello")

    def run(self):
        print(self.requires().foo)  # prints "hello"

This makes it possible to instantiate multiple tasks, eg MyTask(foo='bar') and MyTask(foo='baz'). The task will then have the foo attribute set appropriately.

When a task is instantiated, it will first use any argument as the value of the parameter, eg. if you instantiate a = TaskA(x=44) then a.x == 44. When the value is not provided, the value will be resolved in this order of falling priority:

Parameter objects may be reused, but you must then set the positional=False flag.

years#

__init__(*args, cls=luigi.Parameter, inst=None, unique=False, sort=False, min_len=None,: max_len=None, choices=None, brace_expand=False, escape_sep=True, force_tuple=True, **kwargs)

Parameter that parses a comma-separated value (CSV) and produces a tuple. cls (inst) can refer to an other parameter class (instance) that will be used to parse and serialize the particular items.

When unique is True, both parsing and serialization methods make sure that values are unique. sort can be a boolean or a function for sorting parameter values.

When min_len (max_len) is set to an integer, an error is raised in case the number of elements to serialize or parse (evaluated after potentially ensuring uniqueness) deceeds (exceeds) that value. Just like in luigi’s ChoiceParamater, choices can be a sequence of accepted values.

When brace_expand is True, brace expansion is applied, potentially extending the list of values. However, note that in this case commas that are not meant to act as a delimiter cannot be quoted in csv-style with double quotes, but they should rather be backslash-escaped instead. Unless escape_sep is False, escaped separators (comma) are not split when parsing strings and, likewise, separators contained in values to serialze are escaped.

By default, single values are parsed such that they result in a tuple containing a single item. However, when force_tuple is False, single values that do not end with a comma are not wrapped by a tuple. Likewise, during serialization they are converted to a string as is, whereas tuple containing only a single item will end with a trailing comma.

Example:

p = CSVParameter(cls=luigi.IntParameter)
p.parse("4,5,6,6")
# => (4, 5, 6, 6)
p.serialize((7, 8, 9))
# => "7,8,9"

# "," that should not be used as delimiter
p = CSVParameter()
p.parse("a,b,\"c,d\"")
# -> ("a", "b", "c,d")
# same as
p.parse("a,b,c\,d")
# -> ("a", "b", "c,d")

# uniqueness check
p = CSVParameter(cls=luigi.IntParameter, unique=True)
p.parse("4,5,6,6")
# => (4, 5, 6)

# length check
p = CSVParameter(cls=luigi.IntParameter, max_len=2)
p.parse("4,5,6")
# => ValueError

# choices
p = CSVParameter(cls=luigi.IntParameter, choices=(1, 2))
p.parse("2,3")
# => ValueError

# brace expansion
p = CSVParameter(cls=luigi.IntParameter, brace_expand=True)
# (note that with brace_expand enabled, the quoting if "," only works with back slashes)
p.parse("1{2,3,4}9")
# => (129, 139, 149)

# do not force tuples to wrap single values
p = CSVParameter(cls=luigi.IntParameter, force_tuple=False)
p.parse("1")
# => 1
# note: the result would be (1,) with force_tuple left at True (default)
p.parse("1,")
# => (1,)
p.serialize(1)
# => "1"
p.serialize((1,))
# => "1,"
p.serialize((1, 2))
# => "1,2"

Note

Due to the way instance caching is implemented in luigi, parameters should always have hashable, immutable values. Therefore, this parameter produces a tuple and, in particular, not a list. To avoid undesired side effects, the default value given to the constructor is also converted to a tuple.

_inst#

type: cls

Instance of the luigi parameter class cls or inst directory, that is used internally for parameter parsing and serialization.

class pocket_coffea.law_tasks.configuration.general.datasetconfig(*args, **kwargs)#

Bases: Config

Paramters for dataset creation

allowlist_sites#

__init__(*args, cls=luigi.Parameter, inst=None, unique=False, sort=False, min_len=None,: max_len=None, choices=None, brace_expand=False, escape_sep=True, force_tuple=True, **kwargs)

Parameter that parses a comma-separated value (CSV) and produces a tuple. cls (inst) can refer to an other parameter class (instance) that will be used to parse and serialize the particular items.

When unique is True, both parsing and serialization methods make sure that values are unique. sort can be a boolean or a function for sorting parameter values.

When min_len (max_len) is set to an integer, an error is raised in case the number of elements to serialize or parse (evaluated after potentially ensuring uniqueness) deceeds (exceeds) that value. Just like in luigi’s ChoiceParamater, choices can be a sequence of accepted values.

When brace_expand is True, brace expansion is applied, potentially extending the list of values. However, note that in this case commas that are not meant to act as a delimiter cannot be quoted in csv-style with double quotes, but they should rather be backslash-escaped instead. Unless escape_sep is False, escaped separators (comma) are not split when parsing strings and, likewise, separators contained in values to serialze are escaped.

By default, single values are parsed such that they result in a tuple containing a single item. However, when force_tuple is False, single values that do not end with a comma are not wrapped by a tuple. Likewise, during serialization they are converted to a string as is, whereas tuple containing only a single item will end with a trailing comma.

Example:

p = CSVParameter(cls=luigi.IntParameter)
p.parse("4,5,6,6")
# => (4, 5, 6, 6)
p.serialize((7, 8, 9))
# => "7,8,9"

# "," that should not be used as delimiter
p = CSVParameter()
p.parse("a,b,\"c,d\"")
# -> ("a", "b", "c,d")
# same as
p.parse("a,b,c\,d")
# -> ("a", "b", "c,d")

# uniqueness check
p = CSVParameter(cls=luigi.IntParameter, unique=True)
p.parse("4,5,6,6")
# => (4, 5, 6)

# length check
p = CSVParameter(cls=luigi.IntParameter, max_len=2)
p.parse("4,5,6")
# => ValueError

# choices
p = CSVParameter(cls=luigi.IntParameter, choices=(1, 2))
p.parse("2,3")
# => ValueError

# brace expansion
p = CSVParameter(cls=luigi.IntParameter, brace_expand=True)
# (note that with brace_expand enabled, the quoting if "," only works with back slashes)
p.parse("1{2,3,4}9")
# => (129, 139, 149)

# do not force tuples to wrap single values
p = CSVParameter(cls=luigi.IntParameter, force_tuple=False)
p.parse("1")
# => 1
# note: the result would be (1,) with force_tuple left at True (default)
p.parse("1,")
# => (1,)
p.serialize(1)
# => "1"
p.serialize((1,))
# => "1,"
p.serialize((1, 2))
# => "1,2"

Note

Due to the way instance caching is implemented in luigi, parameters should always have hashable, immutable values. Therefore, this parameter produces a tuple and, in particular, not a list. To avoid undesired side effects, the default value given to the constructor is also converted to a tuple.

_inst#

type: cls

Instance of the luigi parameter class cls or inst directory, that is used internally for parameter parsing and serialization.

blocklist_sites#

__init__(*args, cls=luigi.Parameter, inst=None, unique=False, sort=False, min_len=None,: max_len=None, choices=None, brace_expand=False, escape_sep=True, force_tuple=True, **kwargs)

Parameter that parses a comma-separated value (CSV) and produces a tuple. cls (inst) can refer to an other parameter class (instance) that will be used to parse and serialize the particular items.

When unique is True, both parsing and serialization methods make sure that values are unique. sort can be a boolean or a function for sorting parameter values.

When min_len (max_len) is set to an integer, an error is raised in case the number of elements to serialize or parse (evaluated after potentially ensuring uniqueness) deceeds (exceeds) that value. Just like in luigi’s ChoiceParamater, choices can be a sequence of accepted values.

When brace_expand is True, brace expansion is applied, potentially extending the list of values. However, note that in this case commas that are not meant to act as a delimiter cannot be quoted in csv-style with double quotes, but they should rather be backslash-escaped instead. Unless escape_sep is False, escaped separators (comma) are not split when parsing strings and, likewise, separators contained in values to serialze are escaped.

By default, single values are parsed such that they result in a tuple containing a single item. However, when force_tuple is False, single values that do not end with a comma are not wrapped by a tuple. Likewise, during serialization they are converted to a string as is, whereas tuple containing only a single item will end with a trailing comma.

Example:

p = CSVParameter(cls=luigi.IntParameter)
p.parse("4,5,6,6")
# => (4, 5, 6, 6)
p.serialize((7, 8, 9))
# => "7,8,9"

# "," that should not be used as delimiter
p = CSVParameter()
p.parse("a,b,\"c,d\"")
# -> ("a", "b", "c,d")
# same as
p.parse("a,b,c\,d")
# -> ("a", "b", "c,d")

# uniqueness check
p = CSVParameter(cls=luigi.IntParameter, unique=True)
p.parse("4,5,6,6")
# => (4, 5, 6)

# length check
p = CSVParameter(cls=luigi.IntParameter, max_len=2)
p.parse("4,5,6")
# => ValueError

# choices
p = CSVParameter(cls=luigi.IntParameter, choices=(1, 2))
p.parse("2,3")
# => ValueError

# brace expansion
p = CSVParameter(cls=luigi.IntParameter, brace_expand=True)
# (note that with brace_expand enabled, the quoting if "," only works with back slashes)
p.parse("1{2,3,4}9")
# => (129, 139, 149)

# do not force tuples to wrap single values
p = CSVParameter(cls=luigi.IntParameter, force_tuple=False)
p.parse("1")
# => 1
# note: the result would be (1,) with force_tuple left at True (default)
p.parse("1,")
# => (1,)
p.serialize(1)
# => "1"
p.serialize((1,))
# => "1,"
p.serialize((1, 2))
# => "1,2"

Note

Due to the way instance caching is implemented in luigi, parameters should always have hashable, immutable values. Therefore, this parameter produces a tuple and, in particular, not a list. To avoid undesired side effects, the default value given to the constructor is also converted to a tuple.

_inst#

type: cls

Instance of the luigi parameter class cls or inst directory, that is used internally for parameter parsing and serialization.

check#

A Parameter whose value is a bool. This parameter has an implicit default value of False. For the command line interface this means that the value is False unless you add "--the-bool-parameter" to your command without giving a parameter value. This is considered implicit parsing (the default). However, in some situations one might want to give the explicit bool value ("--the-bool-parameter true|false"), e.g. when you configure the default value to be True. This is called explicit parsing. When omitting the parameter value, it is still considered True but to avoid ambiguities during argument parsing, make sure to always place bool parameters behind the task family on the command line when using explicit parsing.

You can toggle between the two parsing modes on a per-parameter base via

class MyTask(luigi.Task):
    implicit_bool = luigi.BoolParameter(parsing=luigi.BoolParameter.IMPLICIT_PARSING)
    explicit_bool = luigi.BoolParameter(parsing=luigi.BoolParameter.EXPLICIT_PARSING)

or globally by

luigi.BoolParameter.parsing = luigi.BoolParameter.EXPLICIT_PARSING

for all bool parameters instantiated after this line.

dataset_definition#

__init__(*args, cls=luigi.Parameter, inst=None, unique=False, sort=False, min_len=None,: max_len=None, choices=None, brace_expand=False, escape_sep=True, force_tuple=True, **kwargs)

Parameter that parses a comma-separated value (CSV) and produces a tuple. cls (inst) can refer to an other parameter class (instance) that will be used to parse and serialize the particular items.

When unique is True, both parsing and serialization methods make sure that values are unique. sort can be a boolean or a function for sorting parameter values.

When min_len (max_len) is set to an integer, an error is raised in case the number of elements to serialize or parse (evaluated after potentially ensuring uniqueness) deceeds (exceeds) that value. Just like in luigi’s ChoiceParamater, choices can be a sequence of accepted values.

When brace_expand is True, brace expansion is applied, potentially extending the list of values. However, note that in this case commas that are not meant to act as a delimiter cannot be quoted in csv-style with double quotes, but they should rather be backslash-escaped instead. Unless escape_sep is False, escaped separators (comma) are not split when parsing strings and, likewise, separators contained in values to serialze are escaped.

By default, single values are parsed such that they result in a tuple containing a single item. However, when force_tuple is False, single values that do not end with a comma are not wrapped by a tuple. Likewise, during serialization they are converted to a string as is, whereas tuple containing only a single item will end with a trailing comma.

Example:

p = CSVParameter(cls=luigi.IntParameter)
p.parse("4,5,6,6")
# => (4, 5, 6, 6)
p.serialize((7, 8, 9))
# => "7,8,9"

# "," that should not be used as delimiter
p = CSVParameter()
p.parse("a,b,\"c,d\"")
# -> ("a", "b", "c,d")
# same as
p.parse("a,b,c\,d")
# -> ("a", "b", "c,d")

# uniqueness check
p = CSVParameter(cls=luigi.IntParameter, unique=True)
p.parse("4,5,6,6")
# => (4, 5, 6)

# length check
p = CSVParameter(cls=luigi.IntParameter, max_len=2)
p.parse("4,5,6")
# => ValueError

# choices
p = CSVParameter(cls=luigi.IntParameter, choices=(1, 2))
p.parse("2,3")
# => ValueError

# brace expansion
p = CSVParameter(cls=luigi.IntParameter, brace_expand=True)
# (note that with brace_expand enabled, the quoting if "," only works with back slashes)
p.parse("1{2,3,4}9")
# => (129, 139, 149)

# do not force tuples to wrap single values
p = CSVParameter(cls=luigi.IntParameter, force_tuple=False)
p.parse("1")
# => 1
# note: the result would be (1,) with force_tuple left at True (default)
p.parse("1,")
# => (1,)
p.serialize(1)
# => "1"
p.serialize((1,))
# => "1,"
p.serialize((1, 2))
# => "1,2"

Note

Due to the way instance caching is implemented in luigi, parameters should always have hashable, immutable values. Therefore, this parameter produces a tuple and, in particular, not a list. To avoid undesired side effects, the default value given to the constructor is also converted to a tuple.

_inst#

type: cls

Instance of the luigi parameter class cls or inst directory, that is used internally for parameter parsing and serialization.

dataset_dir#

Parameter whose value is a str, and a base class for other parameter types.

Parameters are objects set on the Task class level to make it possible to parameterize tasks. For instance:

class MyTask(luigi.Task):
    foo = luigi.Parameter()

class RequiringTask(luigi.Task):
    def requires(self):
        return MyTask(foo="hello")

    def run(self):
        print(self.requires().foo)  # prints "hello"

This makes it possible to instantiate multiple tasks, eg MyTask(foo='bar') and MyTask(foo='baz'). The task will then have the foo attribute set appropriately.

When a task is instantiated, it will first use any argument as the value of the parameter, eg. if you instantiate a = TaskA(x=44) then a.x == 44. When the value is not provided, the value will be resolved in this order of falling priority:

Parameter objects may be reused, but you must then set the positional=False flag.

download#

A Parameter whose value is a bool. This parameter has an implicit default value of False. For the command line interface this means that the value is False unless you add "--the-bool-parameter" to your command without giving a parameter value. This is considered implicit parsing (the default). However, in some situations one might want to give the explicit bool value ("--the-bool-parameter true|false"), e.g. when you configure the default value to be True. This is called explicit parsing. When omitting the parameter value, it is still considered True but to avoid ambiguities during argument parsing, make sure to always place bool parameters behind the task family on the command line when using explicit parsing.

You can toggle between the two parsing modes on a per-parameter base via

class MyTask(luigi.Task):
    implicit_bool = luigi.BoolParameter(parsing=luigi.BoolParameter.IMPLICIT_PARSING)
    explicit_bool = luigi.BoolParameter(parsing=luigi.BoolParameter.EXPLICIT_PARSING)

or globally by

luigi.BoolParameter.parsing = luigi.BoolParameter.EXPLICIT_PARSING

for all bool parameters instantiated after this line.

include_redirector#

A Parameter whose value is a bool. This parameter has an implicit default value of False. For the command line interface this means that the value is False unless you add "--the-bool-parameter" to your command without giving a parameter value. This is considered implicit parsing (the default). However, in some situations one might want to give the explicit bool value ("--the-bool-parameter true|false"), e.g. when you configure the default value to be True. This is called explicit parsing. When omitting the parameter value, it is still considered True but to avoid ambiguities during argument parsing, make sure to always place bool parameters behind the task family on the command line when using explicit parsing.

You can toggle between the two parsing modes on a per-parameter base via

class MyTask(luigi.Task):
    implicit_bool = luigi.BoolParameter(parsing=luigi.BoolParameter.IMPLICIT_PARSING)
    explicit_bool = luigi.BoolParameter(parsing=luigi.BoolParameter.EXPLICIT_PARSING)

or globally by

luigi.BoolParameter.parsing = luigi.BoolParameter.EXPLICIT_PARSING

for all bool parameters instantiated after this line.

keys#

Parameter whose value is a tuple or tuple of tuples.

In the task definition, use

class MyTask(luigi.Task):
  book_locations = luigi.TupleParameter()

    def run(self):
        for location in self.book_locations:
            print("Go to page %d, line %d" % (location[0], location[1]))

At the command line, use

$ luigi --module my_tasks MyTask --book_locations <JSON string>

Simple example with two grades:

$ luigi --module my_tasks MyTask --book_locations '((12,3),(4,15),(52,1))'

local_prefix#

Parameter whose value is a str, and a base class for other parameter types.

Parameters are objects set on the Task class level to make it possible to parameterize tasks. For instance:

class MyTask(luigi.Task):
    foo = luigi.Parameter()

class RequiringTask(luigi.Task):
    def requires(self):
        return MyTask(foo="hello")

    def run(self):
        print(self.requires().foo)  # prints "hello"

This makes it possible to instantiate multiple tasks, eg MyTask(foo='bar') and MyTask(foo='baz'). The task will then have the foo attribute set appropriately.

When a task is instantiated, it will first use any argument as the value of the parameter, eg. if you instantiate a = TaskA(x=44) then a.x == 44. When the value is not provided, the value will be resolved in this order of falling priority:

Parameter objects may be reused, but you must then set the positional=False flag.

overwrite#

A Parameter whose value is a bool. This parameter has an implicit default value of False. For the command line interface this means that the value is False unless you add "--the-bool-parameter" to your command without giving a parameter value. This is considered implicit parsing (the default). However, in some situations one might want to give the explicit bool value ("--the-bool-parameter true|false"), e.g. when you configure the default value to be True. This is called explicit parsing. When omitting the parameter value, it is still considered True but to avoid ambiguities during argument parsing, make sure to always place bool parameters behind the task family on the command line when using explicit parsing.

You can toggle between the two parsing modes on a per-parameter base via

class MyTask(luigi.Task):
    implicit_bool = luigi.BoolParameter(parsing=luigi.BoolParameter.IMPLICIT_PARSING)
    explicit_bool = luigi.BoolParameter(parsing=luigi.BoolParameter.EXPLICIT_PARSING)

or globally by

luigi.BoolParameter.parsing = luigi.BoolParameter.EXPLICIT_PARSING

for all bool parameters instantiated after this line.

parallelize#: Parameter whose value is an int.

prioritylist_sites#

__init__(*args, cls=luigi.Parameter, inst=None, unique=False, sort=False, min_len=None,: max_len=None, choices=None, brace_expand=False, escape_sep=True, force_tuple=True, **kwargs)

Parameter that parses a comma-separated value (CSV) and produces a tuple. cls (inst) can refer to an other parameter class (instance) that will be used to parse and serialize the particular items.

When unique is True, both parsing and serialization methods make sure that values are unique. sort can be a boolean or a function for sorting parameter values.

When min_len (max_len) is set to an integer, an error is raised in case the number of elements to serialize or parse (evaluated after potentially ensuring uniqueness) deceeds (exceeds) that value. Just like in luigi’s ChoiceParamater, choices can be a sequence of accepted values.

When brace_expand is True, brace expansion is applied, potentially extending the list of values. However, note that in this case commas that are not meant to act as a delimiter cannot be quoted in csv-style with double quotes, but they should rather be backslash-escaped instead. Unless escape_sep is False, escaped separators (comma) are not split when parsing strings and, likewise, separators contained in values to serialze are escaped.

By default, single values are parsed such that they result in a tuple containing a single item. However, when force_tuple is False, single values that do not end with a comma are not wrapped by a tuple. Likewise, during serialization they are converted to a string as is, whereas tuple containing only a single item will end with a trailing comma.

Example:

p = CSVParameter(cls=luigi.IntParameter)
p.parse("4,5,6,6")
# => (4, 5, 6, 6)
p.serialize((7, 8, 9))
# => "7,8,9"

# "," that should not be used as delimiter
p = CSVParameter()
p.parse("a,b,\"c,d\"")
# -> ("a", "b", "c,d")
# same as
p.parse("a,b,c\,d")
# -> ("a", "b", "c,d")

# uniqueness check
p = CSVParameter(cls=luigi.IntParameter, unique=True)
p.parse("4,5,6,6")
# => (4, 5, 6)

# length check
p = CSVParameter(cls=luigi.IntParameter, max_len=2)
p.parse("4,5,6")
# => ValueError

# choices
p = CSVParameter(cls=luigi.IntParameter, choices=(1, 2))
p.parse("2,3")
# => ValueError

# brace expansion
p = CSVParameter(cls=luigi.IntParameter, brace_expand=True)
# (note that with brace_expand enabled, the quoting if "," only works with back slashes)
p.parse("1{2,3,4}9")
# => (129, 139, 149)

# do not force tuples to wrap single values
p = CSVParameter(cls=luigi.IntParameter, force_tuple=False)
p.parse("1")
# => 1
# note: the result would be (1,) with force_tuple left at True (default)
p.parse("1,")
# => (1,)
p.serialize(1)
# => "1"
p.serialize((1,))
# => "1,"
p.serialize((1, 2))
# => "1,2"

Note

Due to the way instance caching is implemented in luigi, parameters should always have hashable, immutable values. Therefore, this parameter produces a tuple and, in particular, not a list. To avoid undesired side effects, the default value given to the constructor is also converted to a tuple.

_inst#

type: cls

Instance of the luigi parameter class cls or inst directory, that is used internally for parameter parsing and serialization.

regex_sites#

Parameter whose value is a str, and a base class for other parameter types.

Parameters are objects set on the Task class level to make it possible to parameterize tasks. For instance:

class MyTask(luigi.Task):
    foo = luigi.Parameter()

class RequiringTask(luigi.Task):
    def requires(self):
        return MyTask(foo="hello")

    def run(self):
        print(self.requires().foo)  # prints "hello"

This makes it possible to instantiate multiple tasks, eg MyTask(foo='bar') and MyTask(foo='baz'). The task will then have the foo attribute set appropriately.

When a task is instantiated, it will first use any argument as the value of the parameter, eg. if you instantiate a = TaskA(x=44) then a.x == 44. When the value is not provided, the value will be resolved in this order of falling priority:

Parameter objects may be reused, but you must then set the positional=False flag.

sort_replicas#

Parameter whose value is a str, and a base class for other parameter types.

Parameters are objects set on the Task class level to make it possible to parameterize tasks. For instance:

class MyTask(luigi.Task):
    foo = luigi.Parameter()

class RequiringTask(luigi.Task):
    def requires(self):
        return MyTask(foo="hello")

    def run(self):
        print(self.requires().foo)  # prints "hello"

This makes it possible to instantiate multiple tasks, eg MyTask(foo='bar') and MyTask(foo='baz'). The task will then have the foo attribute set appropriately.

When a task is instantiated, it will first use any argument as the value of the parameter, eg. if you instantiate a = TaskA(x=44) then a.x == 44. When the value is not provided, the value will be resolved in this order of falling priority:

Parameter objects may be reused, but you must then set the positional=False flag.

split_by_year#

A Parameter whose value is a bool. This parameter has an implicit default value of False. For the command line interface this means that the value is False unless you add "--the-bool-parameter" to your command without giving a parameter value. This is considered implicit parsing (the default). However, in some situations one might want to give the explicit bool value ("--the-bool-parameter true|false"), e.g. when you configure the default value to be True. This is called explicit parsing. When omitting the parameter value, it is still considered True but to avoid ambiguities during argument parsing, make sure to always place bool parameters behind the task family on the command line when using explicit parsing.

You can toggle between the two parsing modes on a per-parameter base via

class MyTask(luigi.Task):
    implicit_bool = luigi.BoolParameter(parsing=luigi.BoolParameter.IMPLICIT_PARSING)
    explicit_bool = luigi.BoolParameter(parsing=luigi.BoolParameter.EXPLICIT_PARSING)

or globally by

luigi.BoolParameter.parsing = luigi.BoolParameter.EXPLICIT_PARSING

for all bool parameters instantiated after this line.

class pocket_coffea.law_tasks.configuration.general.runnerconfig(*args, **kwargs)#

Bases: Config

coffea_output#

Parameter whose value is a str, and a base class for other parameter types.

Parameters are objects set on the Task class level to make it possible to parameterize tasks. For instance:

class MyTask(luigi.Task):
    foo = luigi.Parameter()

class RequiringTask(luigi.Task):
    def requires(self):
        return MyTask(foo="hello")

    def run(self):
        print(self.requires().foo)  # prints "hello"

This makes it possible to instantiate multiple tasks, eg MyTask(foo='bar') and MyTask(foo='baz'). The task will then have the foo attribute set appropriately.

When a task is instantiated, it will first use any argument as the value of the parameter, eg. if you instantiate a = TaskA(x=44) then a.x == 44. When the value is not provided, the value will be resolved in this order of falling priority:

Parameter objects may be reused, but you must then set the positional=False flag.

executor#

Parameter whose value is a str, and a base class for other parameter types.

Parameters are objects set on the Task class level to make it possible to parameterize tasks. For instance:

class MyTask(luigi.Task):
    foo = luigi.Parameter()

class RequiringTask(luigi.Task):
    def requires(self):
        return MyTask(foo="hello")

    def run(self):
        print(self.requires().foo)  # prints "hello"

This makes it possible to instantiate multiple tasks, eg MyTask(foo='bar') and MyTask(foo='baz'). The task will then have the foo attribute set appropriately.

When a task is instantiated, it will first use any argument as the value of the parameter, eg. if you instantiate a = TaskA(x=44) then a.x == 44. When the value is not provided, the value will be resolved in this order of falling priority:

Parameter objects may be reused, but you must then set the positional=False flag.

limit_chunks#: Parameter whose value is an int.

limit_files#: Parameter whose value is an int.

process_separately#

A Parameter whose value is a bool. This parameter has an implicit default value of False. For the command line interface this means that the value is False unless you add "--the-bool-parameter" to your command without giving a parameter value. This is considered implicit parsing (the default). However, in some situations one might want to give the explicit bool value ("--the-bool-parameter true|false"), e.g. when you configure the default value to be True. This is called explicit parsing. When omitting the parameter value, it is still considered True but to avoid ambiguities during argument parsing, make sure to always place bool parameters behind the task family on the command line when using explicit parsing.

You can toggle between the two parsing modes on a per-parameter base via

class MyTask(luigi.Task):
    implicit_bool = luigi.BoolParameter(parsing=luigi.BoolParameter.IMPLICIT_PARSING)
    explicit_bool = luigi.BoolParameter(parsing=luigi.BoolParameter.EXPLICIT_PARSING)

or globally by

luigi.BoolParameter.parsing = luigi.BoolParameter.EXPLICIT_PARSING

for all bool parameters instantiated after this line.

scaleout#: Parameter whose value is an int.

skip_bad_files#

A Parameter whose value is a bool. This parameter has an implicit default value of False. For the command line interface this means that the value is False unless you add "--the-bool-parameter" to your command without giving a parameter value. This is considered implicit parsing (the default). However, in some situations one might want to give the explicit bool value ("--the-bool-parameter true|false"), e.g. when you configure the default value to be True. This is called explicit parsing. When omitting the parameter value, it is still considered True but to avoid ambiguities during argument parsing, make sure to always place bool parameters behind the task family on the command line when using explicit parsing.

You can toggle between the two parsing modes on a per-parameter base via

class MyTask(luigi.Task):
    implicit_bool = luigi.BoolParameter(parsing=luigi.BoolParameter.IMPLICIT_PARSING)
    explicit_bool = luigi.BoolParameter(parsing=luigi.BoolParameter.EXPLICIT_PARSING)

or globally by

luigi.BoolParameter.parsing = luigi.BoolParameter.EXPLICIT_PARSING

for all bool parameters instantiated after this line.

test#

A Parameter whose value is a bool. This parameter has an implicit default value of False. For the command line interface this means that the value is False unless you add "--the-bool-parameter" to your command without giving a parameter value. This is considered implicit parsing (the default). However, in some situations one might want to give the explicit bool value ("--the-bool-parameter true|false"), e.g. when you configure the default value to be True. This is called explicit parsing. When omitting the parameter value, it is still considered True but to avoid ambiguities during argument parsing, make sure to always place bool parameters behind the task family on the command line when using explicit parsing.

You can toggle between the two parsing modes on a per-parameter base via

class MyTask(luigi.Task):
    implicit_bool = luigi.BoolParameter(parsing=luigi.BoolParameter.IMPLICIT_PARSING)
    explicit_bool = luigi.BoolParameter(parsing=luigi.BoolParameter.EXPLICIT_PARSING)

or globally by

luigi.BoolParameter.parsing = luigi.BoolParameter.EXPLICIT_PARSING

for all bool parameters instantiated after this line.

class pocket_coffea.law_tasks.configuration.general.transferconfig(*args, **kwargs)#

Bases: Config

Config class for transfer to wlcg.

transfer#

A Parameter whose value is a bool. This parameter has an implicit default value of False. For the command line interface this means that the value is False unless you add "--the-bool-parameter" to your command without giving a parameter value. This is considered implicit parsing (the default). However, in some situations one might want to give the explicit bool value ("--the-bool-parameter true|false"), e.g. when you configure the default value to be True. This is called explicit parsing. When omitting the parameter value, it is still considered True but to avoid ambiguities during argument parsing, make sure to always place bool parameters behind the task family on the command line when using explicit parsing.

You can toggle between the two parsing modes on a per-parameter base via

class MyTask(luigi.Task):
    implicit_bool = luigi.BoolParameter(parsing=luigi.BoolParameter.IMPLICIT_PARSING)
    explicit_bool = luigi.BoolParameter(parsing=luigi.BoolParameter.EXPLICIT_PARSING)

or globally by

luigi.BoolParameter.parsing = luigi.BoolParameter.EXPLICIT_PARSING

for all bool parameters instantiated after this line.

pocket_coffea.law_tasks.configuration package

Contents

pocket_coffea.law_tasks.configuration package#

Submodules#

pocket_coffea.law_tasks.configuration.general module#

pocket_coffea.law_tasks.configuration.plotting module#

Module contents#