c_preproc¶
C/C++ preprocessor for finding dependencies
Reasons for using the Waf preprocessor by default
- Some c/c++ extensions (Qt) require a custom preprocessor for obtaining the dependencies (.moc files)
- Not all compilers provide .d files for obtaining the dependencies (portability)
- A naive file scanner will not catch the constructs such as “#include foo()”
- A naive file scanner will catch unnecessary dependencies (change an unused header -> recompile everything)
Regarding the speed concerns:
- the preprocessing is performed only when files must be compiled
- the macros are evaluated only for #if/#elif/#include
- system headers are not scanned by default
Now if you do not want the Waf preprocessor, the tool +gccdeps* uses the .d files produced during the compilation to track the dependencies (useful when used with the boost libraries). It only works with gcc >= 4.4 though.
A dumb preprocessor is also available in the tool c_dumbpreproc
-
exception
waflib.Tools.c_preproc.
PreprocError
(msg='', ex=None)[source]¶ Bases:
waflib.Errors.WafError
-
waflib.Tools.c_preproc.
POPFILE
= '-'¶ Constant representing a special token used in
waflib.Tools.c_preproc.c_parser.start()
iteration to switch to a header read previously
-
waflib.Tools.c_preproc.
recursion_limit
= 150¶ Limit on the amount of files to read in the dependency scanner
-
waflib.Tools.c_preproc.
go_absolute
= False¶ Set to True to track headers on files in /usr/include, else absolute paths are ignored (but it becomes very slow)
-
waflib.Tools.c_preproc.
use_trigraphs
= 0¶ Apply trigraph rules (False by default)
-
waflib.Tools.c_preproc.
g_optrans
= {'and': '&&', 'and_eq': '&=', 'bitand': '&', 'bitor': '|', 'compl': '~', 'not': '!', 'not_eq': '!', 'or': '||', 'or_eq': '|=', 'xor': '^', 'xor_eq': '^='}¶ Operators such as and/or/xor for c++. Set an empty dict to disable.
-
waflib.Tools.c_preproc.
re_lines
= re.compile('^[ \t]*(?:#|%:)[ \t]*(ifdef|ifndef|if|else|elif|endif|include|import|define|undef|pragma)[ \t]*(.*)\r*$', re.IGNORECASE|re.MULTILINE)¶ Match #include lines
-
waflib.Tools.c_preproc.
re_mac
= re.compile('^[a-zA-Z_]\\w*')¶ Match macro definitions
-
waflib.Tools.c_preproc.
re_fun
= re.compile('^[a-zA-Z_][a-zA-Z0-9_]*[(]')¶ Match macro functions
-
waflib.Tools.c_preproc.
re_pragma_once
= re.compile('^\\s*once\\s*', re.IGNORECASE)¶ Match #pragma once statements
-
waflib.Tools.c_preproc.
re_nl
= re.compile('\\\\\r*\n', re.MULTILINE)¶ Match newlines
-
waflib.Tools.c_preproc.
re_cpp
= re.compile('//.*?$|/\\*.*?\\*/|\\\'(?:\\\\.|[^\\\\\\\'])*\\\'|"(?:\\\\.|[^\\\\"])*"', re.MULTILINE|re.DOTALL)¶ Filter C/C++ comments
-
waflib.Tools.c_preproc.
trig_def
= [('??=', '#'), ('??-', '~'), ('??/', '\\'), ('??!', '|'), ("??'", '^'), ('??(', '['), ('??)', ']'), ('??<', '{'), ('??>', '}')]¶ Trigraph definitions
-
waflib.Tools.c_preproc.
chr_esc
= {"'": 39, '0': 0, '\\': 92, 'a': 7, 'b': 8, 'f': 11, 'n': 10, 'r': 13, 't': 9, 'v': 12}¶ Escape characters
-
waflib.Tools.c_preproc.
NUM
= 'i'¶ Number token
-
waflib.Tools.c_preproc.
OP
= 'O'¶ Operator token
-
waflib.Tools.c_preproc.
IDENT
= 'T'¶ Identifier token
-
waflib.Tools.c_preproc.
STR
= 's'¶ String token
-
waflib.Tools.c_preproc.
CHAR
= 'c'¶ Character token
-
waflib.Tools.c_preproc.
tok_types
= ['i', 's', 'T', 'O']¶ Token types
-
waflib.Tools.c_preproc.
exp_types
= ["0[xX](?P<hex>[a-fA-F0-9]+)(?P<qual1>[uUlL]*)|L*?'(?P<char>(\\\\.|[^\\\\'])+)'|(?P<n1>\\d+)[Ee](?P<exp0>[+-]*?\\d+)(?P<float0>[fFlL]*)|(?P<n2>\\d*\\.\\d+)([Ee](?P<exp1>[+-]*?\\d+))?(?P<float1>[fFlL]*)|(?P<n4>\\d+\\.\\d*)([Ee](?P<exp2>[+-]*?\\d+))?(?P<float2>[fFlL]*)|(?P<oct>0*)(?P<n0>\\d+)(?P<qual2>[uUlL]*)", 'L?"([^"\\\\]|\\\\.)*"', '[a-zA-Z_]\\w*', '%:%:|<<=|>>=|\\.\\.\\.|<<|<%|<:|<=|>>|>=|\\+\\+|\\+=|--|->|-=|\\*=|/=|%:|%=|%>|==|&&|&=|\\|\\||\\|=|\\^=|:>|!=|##|[\\(\\)\\{\\}\\[\\]<>\\?\\|\\^\\*\\+&=:!#;,%/\\-\\?\\~\\.]']¶ Expression types
-
waflib.Tools.c_preproc.
re_clexer
= re.compile('(?P<i>0[xX](?P<hex>[a-fA-F0-9]+)(?P<qual1>[uUlL]*)|L*?\'(?P<char>(\\\\.|[^\\\\\'])+)\'|(?P<n1>\\d+)[Ee](?P<exp0>[+-]*?\\d+)(?P<float0>[fFlL]*)|(?P<n2>\\d*\\.\\d+)([Ee](?P<exp1>[+-]*?\\d+))?(?P<float1, re.MULTILINE)¶ Match expressions into tokens
-
waflib.Tools.c_preproc.
accepted
= 'a'¶ Parser state is accepted
-
waflib.Tools.c_preproc.
ignored
= 'i'¶ Parser state is ignored, for example preprocessor lines in an #if 0 block
-
waflib.Tools.c_preproc.
undefined
= 'u'¶ Parser state is undefined at the moment
-
waflib.Tools.c_preproc.
skipped
= 's'¶ Parser state is skipped, for example preprocessor lines in a #elif 0 block
-
waflib.Tools.c_preproc.
repl
(m)[source]¶ Replace function used with
waflib.Tools.c_preproc.re_cpp
-
waflib.Tools.c_preproc.
prec
= {'!=': 4, '%': 0, '&': 5, '&&': 6, '*': 0, '+': 1, ',': 7, '-': 1, '/': 0, '<': 3, '<<': 2, '<=': 3, '==': 4, '>': 3, '>=': 3, '>>': 2, '^': 5, '|': 5, '||': 6}¶ Operator precedence rules required for parsing expressions of the form:
#if 1 && 2 != 0
-
waflib.Tools.c_preproc.
reduce_nums
(val_1, val_2, val_op)[source]¶ Apply arithmetic rules to compute a result
Parameters: - val1 (int or string) – input parameter
- val2 (int or string) – input parameter
- val_op (string) – C operator in +, /, -, etc
Return type: int
-
waflib.Tools.c_preproc.
get_num
(lst)[source]¶ Try to obtain a number from a list of tokens. The token types are defined in
waflib.Tools.ccroot.tok_types
.Parameters: lst (list of tuple (tokentype, value)) – list of preprocessor tokens Returns: a pair containing the number and the rest of the list Return type: tuple(value, list)
-
waflib.Tools.c_preproc.
get_term
(lst)[source]¶ Evaluate an expression recursively, for example:
1+1+1 -> 2+1 -> 3
Parameters: lst (list of tuple(token, value)) – list of tokens Returns: the value and the remaining tokens Return type: value, list
-
waflib.Tools.c_preproc.
reduce_eval
(lst)[source]¶ Take a list of tokens and output true or false for #if/#elif conditions.
Parameters: lst (list of tuple(token, value)) – a list of tokens Returns: a token Return type: tuple(NUM, int)
-
waflib.Tools.c_preproc.
stringize
(lst)[source]¶ Merge a list of tokens into a string
Parameters: lst (list of tuple(token, value)) – a list of tokens Return type: string
-
waflib.Tools.c_preproc.
paste_tokens
(t1, t2)[source]¶ Token pasting works between identifiers, particular operators, and identifiers and numbers:
a ## b -> ab > ## = -> >= a ## 2 -> a2
Parameters: - t1 (tuple(type, value)) – token
- t2 (tuple(type, value)) – token
-
waflib.Tools.c_preproc.
reduce_tokens
(lst, defs, ban=[])[source]¶ Replace the tokens in lst, using the macros provided in defs, and a list of macros that cannot be re-applied
Parameters: - lst (list of tuple(token, value)) – list of tokens
- defs (dict) – macro definitions
- ban (list of string) – macros that cannot be substituted (recursion is not allowed)
Returns: the new list of tokens
Return type: value, list
-
waflib.Tools.c_preproc.
eval_macro
(lst, defs)[source]¶ Reduce the tokens by
waflib.Tools.c_preproc.reduce_tokens()
and try to return a 0/1 result bywaflib.Tools.c_preproc.reduce_eval()
.Parameters: - lst (list of tuple(token, value)) – list of tokens
- defs (dict) – macro definitions
Return type: int
-
waflib.Tools.c_preproc.
extract_macro
(txt)[source]¶ - Process a macro definition of the form::
- #define f(x, y) x * y
into a function or a simple macro without arguments
Parameters: txt (string) – expression to exact a macro definition from Returns: a tuple containing the name, the list of arguments and the replacement Return type: tuple(string, [list, list])
-
waflib.Tools.c_preproc.
extract_include
(txt, defs)[source]¶ Process a line in the form:
#include foo
Parameters: - txt (string) – include line to process
- defs (dict) – macro definitions
Returns: the file name
Return type: string
-
waflib.Tools.c_preproc.
parse_char
(txt)[source]¶ Parse a c character
Parameters: txt (string) – character to parse Returns: a character literal Return type: string
-
waflib.Tools.c_preproc.
tokenize
(s)[source]¶ Convert a string into a list of tokens (shlex.split does not apply to c/c++/d)
Parameters: s (string) – input to tokenize Returns: a list of tokens Return type: list of tuple(token, value)
-
class
waflib.Tools.c_preproc.
c_parser
(nodepaths=None, defines=None)[source]¶ Bases:
object
Used by
waflib.Tools.c_preproc.scan()
to parse c/h files. Note that by default, only project headers are parsed.-
lines
= None¶ list of lines read
-
nodepaths
= None¶ Include paths
-
nodes
= None¶ List of
waflib.Node.Node
found so far
-
names
= None¶ List of file names that could not be matched by any file
-
curfile
= None¶ Current file
-
ban_includes
= None¶ Includes that must not be read (#pragma once)
-
listed
= None¶ Include nodes/names already listed to avoid duplicates in self.nodes/self.names
-
cached_find_resource
(node, filename)[source]¶ Find a file from the input directory
Parameters: - node (
waflib.Node.Node
) – directory - filename (string) – header to find
Returns: the node if found, or None
Return type: - node (
-
tryfind
(filename, kind='"', env=None)[source]¶ Try to obtain a node from the filename based from the include paths. Will add the node found to
waflib.Tools.c_preproc.c_parser.nodes
or the file name towaflib.Tools.c_preproc.c_parser.names
if no corresponding file is found. Called bywaflib.Tools.c_preproc.c_parser.start
.Parameters: filename (string) – header to find Returns: the node if found Return type: waflib.Node.Node
-
filter_comments
(node)[source]¶ Filter the comments from a c/h file, and return the preprocessor lines. The regexps
waflib.Tools.c_preproc.re_cpp
,waflib.Tools.c_preproc.re_nl
andwaflib.Tools.c_preproc.re_lines
are used internally.Returns: the preprocessor directives as a list of (keyword, line) Return type: a list of string pairs
-
addlines
(node)[source]¶ Add the lines from a header in the list of preprocessor lines to parse
Parameters: node ( waflib.Node.Node
) – header
-
start
(node, env)[source]¶ Preprocess a source file to obtain the dependencies, which are accumulated to
waflib.Tools.c_preproc.c_parser.nodes
andwaflib.Tools.c_preproc.c_parser.names
.Parameters: - node (
waflib.Node.Node
) – source file - env (
waflib.ConfigSet.ConfigSet
) – config set containing additional defines to take into account
- node (
-
-
waflib.Tools.c_preproc.
scan
(task)[source]¶ Get the dependencies using a c/c++ preprocessor, this is required for finding dependencies of the kind:
#include some_macro()
This function is bound as a task method on
waflib.Tools.c.c
andwaflib.Tools.cxx.cxx
for example