:py:mod:`flow.record.selector` ============================== .. py:module:: flow.record.selector Module Contents --------------- Classes ~~~~~~~ .. autoapisummary:: flow.record.selector.NoneObject flow.record.selector.SelectorResult flow.record.selector.Selector flow.record.selector.WrappedRecord flow.record.selector.CompiledSelector flow.record.selector.TypeMatcher flow.record.selector.TypeMatcherInstance flow.record.selector.RecordContextMatcher Functions ~~~~~~~~~ .. autoapisummary:: :nosignatures: flow.record.selector.lower flow.record.selector.upper flow.record.selector.names flow.record.selector.name flow.record.selector.get_type flow.record.selector.has_field flow.record.selector.field_regex flow.record.selector.field_equals flow.record.selector.field_contains flow.record.selector.resolve_attr_path flow.record.selector.make_selector Attributes ~~~~~~~~~~ .. autoapisummary:: flow.record.selector.HAVE_ASTOR flow.record.selector.string_types flow.record.selector.AST_NODE_S_TYPES flow.record.selector.AST_NODE_VALUE_TYPES flow.record.selector.AST_OPERATORS flow.record.selector.AST_COMPARATORS flow.record.selector.NONE_OBJECT flow.record.selector.FUNCTION_WHITELIST .. py:data:: HAVE_ASTOR :value: True .. py:data:: string_types :value: () .. py:data:: AST_NODE_S_TYPES .. py:data:: AST_NODE_VALUE_TYPES .. py:data:: AST_OPERATORS .. py:data:: AST_COMPARATORS .. py:class:: NoneObject Returned in the Selector matching if a field does not exist on the Record. NoneObject is used to override some comparators like __contains__. .. py:method:: __eq__(b) Return self==value. .. py:method:: __ne__(b) Return self!=value. .. py:method:: __lt__(b) Return selfvalue. .. py:method:: __lte__(b) .. py:method:: __gte__(b) .. py:method:: __noteq__(b) .. py:method:: __contains__(b) .. py:method:: __len__() .. py:data:: NONE_OBJECT .. py:exception:: InvalidSelectorError Bases: :py:obj:`Exception` Common base class for all non-exit exceptions. .. py:exception:: InvalidOperation Bases: :py:obj:`Exception` Common base class for all non-exit exceptions. .. py:function:: lower(s) Return lowercased string, otherwise `s` if not string type. .. py:function:: upper(s) Return uppercased string, otherwise `s` if not string type. .. py:function:: names(r) Return the available names as a set in the Record otherwise ['UnknownRecord']. .. py:function:: name(r) Return the name of the Record otherwise 'UnknownRecord'. .. py:function:: get_type(obj) Return the type of the Object as 'str'. .. py:function:: has_field(r, field) Check if field exists on Record object. :param r: Record to match on. :param field_name: Field name :returns: True if field exists, otherwise False :rtype: (bool) .. py:function:: field_regex(r, fields, regex) Check a regex against fields of a Record object. :param r: The record to match on. :param fields: The fields in the Record to match. :param regex: The regex pattern to search for. :returns: True or False :rtype: (bool) .. py:function:: field_equals(r, fields, strings, nocase=True) Check for exact string matches on fields of a Record object. :param r: The record to match on. :param fields: The fields in the Record to match. :param strings: The strings to search for. :param nocase: Should the matching be case insensitive. :returns: True or False :rtype: (bool) .. py:function:: field_contains(r, fields, strings, nocase=True, word_boundary=False) Check if the string matches on fields of a Record object. Only supports strings for now and partial matches using the __contains__ operator. * `fields` is a list of field names to check * `strings` is a list of strings to check on the fields * `word_boundary` is a boolean. True if matching required only word boundary matches. * Non existing fields on the Record object are skipped. * Defaults to case-insensitive matching, use `nocase=False` if you want to be case sensitive. .. py:data:: FUNCTION_WHITELIST .. py:function:: resolve_attr_path(node) Resolve a node attribute to full path, eg: net.ipv4.Subnet. .. py:class:: SelectorResult(expression_str, match_result, backtrace, referenced_fields) .. py:method:: backtrace() .. py:class:: Selector(expression) .. py:attribute:: VERBOSITY_ALL :value: 1 .. py:attribute:: VERBOSITY_BRANCHES :value: 2 .. py:attribute:: VERBOSITY_NONE :value: 3 .. py:method:: __str__() Return str(self). .. py:method:: __repr__() Return repr(self). .. py:method:: __contains__(record) .. py:method:: explain_selector(record, verbosity=VERBOSITY_ALL) .. py:method:: match(record) .. py:class:: WrappedRecord(record) WrappedRecord wraps a Record but will return a NoneObject for non existing attributes. .. py:attribute:: __slots__ :value: ('record',) .. py:method:: __getattr__(k) .. py:method:: __str__() -> str Return str(self). .. py:method:: __repr__() -> str Return repr(self). .. py:class:: CompiledSelector(expression) CompiledSelector is faster than Selector but unsafe if you don't trust the query. .. py:method:: __str__() Return str(self). .. py:method:: __repr__() Return repr(self). .. py:method:: __contains__(record) .. py:method:: match(record) .. py:class:: TypeMatcher(rec) Helper to get and check fields of a certain type. Types can be selected using `Type.`. Attributes can be selected using `Type..`. For example `Type.uri.filename` will retrieve all the filenames from all uri's in a record. These selectors can also still be used in other helper functions, as they will unwrap to resulting fieldnames. So for example, you can still do `field_contains(r, Type.string, ['something'])`, which will check all `string` fields. Membership tests also work. `'something' in Type.string` will perform a membership test in each string value and return True if there are any. Reverse membership tests are trickier, and only work with a non-compiled Selector. For example, `Type.net.ipv4.Address in net.ipv4.Subnet('10.0.0.0/8')` requires the TypeMatcher to unroll its values, which is only possible when overriding this behaviour. .. py:method:: __getattr__(attr) .. py:class:: TypeMatcherInstance(rec, ftypeparts=None, attrs=None) .. py:method:: __getattr__(attr) .. py:method:: __iter__() .. py:method:: __eq__(other) Return self==value. .. py:method:: __ne__(other) Return self!=value. .. py:method:: __lt__(other) Return selfvalue. .. py:method:: __lte__(other) .. py:method:: __gte__(other) .. py:method:: __noteq__(other) .. py:method:: __contains__(other) .. py:class:: RecordContextMatcher(expr, expr_str, backtrace_verbosity=Selector.VERBOSITY_NONE) .. py:method:: matches(rec) .. py:method:: eval(node) .. py:function:: make_selector(selector, force_compiled=False) Return a Selector object (either CompiledSelector or Selector).