|
NAME | C SYNOPSIS | DESCRIPTION | SEMANTIC CHECKS AND RULES | EXPRESSION EVALUATION | CAVEATS | DIAGNOSTICS | SEE ALSO | COLOPHON |
|
PMREGISTERDERIVED(3) Library Functions Manual PMREGISTERDERIVED(3)
pmRegisterDerived, pmRegisterDerivedMetric - register a derived met‐
ric name and definition
#include <pcp/pmapi.h>
char *pmRegisterDerived(char *name, char *expr);
int pmRegisterDerivedMetric(char *name, char *expr, char **errmsg);
cc ... -lpcp
Derived metrics provide a way of extending the Performance Metrics
Name Space (PMNS) with new metrics defined at the PCP client-side
using expressions over the existing performance metrics.
Typical uses would be to aggregate a number of similar metrics to
provide a higher-level summary metric or to support the ``delta V
over delta V'' class of metrics that are not possible in the base
data semantics of PCP. An example of the latter class would be the
average I/O size, defined as
delta(disk.dev.total_bytes) / delta(disk.dev.total)
where both of the disk.dev metrics are counters, and what is required
is to to sample both metrics, compute the difference between the
current and previous values and then calculate the ratio of these
differences.
The arguments to pmRegisterDerived are the name of the new derived
metric and expr is an expression defining how the values of name
should be computed.
pmRegisterDerivedMetric is the exact functional equivalent to
pmRegisterDerived except that it provides a simplified model of error
handling, where a formatted message is returned via the errmsg
parameter.
Syntactic checking is performed at the time pmRegisterDerived is
called, but semantic checking is deferred until each new PMAPI
context is created with pmNewContext(3) or re-established with
pmReconnectContext(3), at which time the PMNS and metadata is
available to allow semantic checking and the metadata of the derived
metrics to be determined. This means pmRegisterDerived does not
apply retrospectively to any open PMAPI contexts, nor to any PMAPI
contexts already open at the time pmRegisterDerived is called, so the
normal use would be to make all calls to pmRegisterDerived (possibly
via pmLoadDerivedConfig(3)) or pmRegisterDerivedMetric and then call
pmNewContext(3).
name should follow the syntactic rules for the names of performance
metrics, namely one or more components separated with a dot (``.''),
and each component must begin with an alphabetic followed by zero or
more characters drawn from the alphabetics, numerics and underscore
(``_''). For more details, refer to PCPIntro(1) and pmns(5).
name must be unique across all derived metrics and should not match
the name of any regular metric in the PMNS. It is acceptable for
name to share some part of its prefix with an existing subtree of the
PMNS, e.g. the average I/O size metric above could be named
disk.dev.avgsz which would place it amongst the other disk.dev
metrics in the PMNS. Alternatively, derived metrics could populate
their own subtree of the PMNS, e.g. the average I/O size metric above
could be named my.summary.disk.avgsz.
The expression expr follows these syntactic rules:
* Terminal elements are either names of existing metrics or numeric
constants. Recursive definitions are not allowed, so only the
names of regular metrics (not other derived metrics) may be used.
Numeric constants are either integers constrained to the precision
of 32-bit unsigned integers or double precision floating point
numbers.
* The usual binary arithmetic operators are supported, namely
addition (``+''), subtraction (``-''), multiplication (``*'') and
division (``/'') with the normal precedence rules where
multiplication and division have higher precedence than addition
and subtraction, so a+b*c is evaluated as a+(b*c).
* Unary negation may be used, e.g. -3*some.metric.
* C-style relational operators are supported, namely ``<'', ``<='',
``=='', ``>='', ``>'' and ``!=''. Relational expresssions return a
value as a 32-bit unsigned number being 0 for false and 1 for true.
The expected operator precedence rules apply, so arithmetic
operators have higher precedence than relational operators, and a-
b>c+d is evaluated as (a-b)>(c+d). All the relational operators
have equal precedence, so the (slightly odd) expression involving
consecutive relational operators a>b!=c is evaluated as (a>b)!=c.
* C-style boolean operators are supported, namely and (``&&'') and or
(``||''). Boolean expresssions return a value as a 32-bit unsigned
number being 0 for false and 1 for true. The expected operator
precedence rules apply, so relational operators have higher
precedence than boolean operators, and a>b*c&&d<=e+f is evaluated
as (a>(b*c))&&(d<=(e+f)). Both the boolean operators have equal
precedence, so the expression involving consecutive boolean
operators a>=b||b>c&&d!=e||f>g is evaluated as
(((a>=b)||(b>c))&&(d!=e))||(f>g).
* Additionally, the ``!'' operator may be used to negate a boolean or
relational expression, returning a value as a 32-bit unsigned
number being 0 for false and 1 for true. The expected operator
precedence rules apply, so boolean (and relational) operators have
higher precedence than boolean negation, and !a>b||c<d is evaluated
as !((a>b)||(c<d)), while !a<b+c is evaluated as !(a<(b+c)).
* C-style ternary conditional expressions are supported. In general
terms the expression check ? foo : bar is evaluated as foo (the
``true'' operand) if check (the ``guard'') is true, else the
expression evaluates to bar (the ``false'' operand). Some special
semantic rules apply to the ``guard'' expression and the other two
operand expressions:
(a) Each expression may involve a singular value or a set of values
(when the expression involves one or more metrics with an
instance domain).
(b) All expressions with a set of values must be defined over the
same instance domain.
(c) Both operand expressions must have the same metadata, so the
same metric type, semantics and units (dimension and scale).
(d) The ``guard'' expression must have an aritmetic or relational
or boolean value, so that it can be evaluated as 0 for false,
else true.
(e) If the ``guard'' expression has a singular value and one or
more of the other operand expressions involves an instance
domain, the ``guard'' applies to all instances.
(f) If the ``guard'' expression has a set of values and one or more
of the other operand expressions involves an instance domain,
the ``guard'' is evaluated once for each instance (there must
be one instance domain as per rule (b) above).
(g) If one of the operand expressions has a singular value and the
other has a set of values, and the singular value is selected
based on the evaluation of the ``guard'', then the result is a
set of values (all the same) with instance enumeration being
taken from the other operand expression. For example in the
expression: foo ? scalar : set, if foo is true, then the result
is a set of values (all having the same value, scalar) over the
instance domain of set.
* Numeric constants can also be specified using the mkconst()
constructor which takes a number of arguments: the first is a
numeric constant (either integer or floating point), then follow
one or more parameters of the form tag=value or tag= where the
allowed values of tag and value are as follows:
┌──────────┬───────────────────────────────────────────────┐
│ tag │ value │
├──────────┼───────────────────────────────────────────────┤
│type │ one of the numeric metric types from │
│ │ <pcp/pmapi.h>, stripped of the PM_TYPE_ │
│ │ prefix, so 32, U32, 64, U64, FLOAT or DOUBLE. │
├──────────┼───────────────────────────────────────────────┤
│semantics │ one of the semantic types from <pcp/pmapi.h>, │
│ │ stripped of the PM_SEM_ prefix, so COUNTER, │
│ │ INSTANT or DISCRETE. │
├──────────┼───────────────────────────────────────────────┤
│units │ a specification of dimension and scale │
│ │ (together forming the units), in the syntax │
│ │ accepted by pmParseUnitsStr(3). │
└──────────┴───────────────────────────────────────────────┘
The value may optionally be enclosed in double quotes, and may
appear in any mix of upper and/or lower case. The tag must be in
lower case as shown in the table above.
This is most useful when the expression semantics require matching
type and/or semantics and/or units for operands, e.g.
idle = mem.util.free > mkconst(10485760, units=Kbyte)
avg_io_size = delta(disk.dev.total) == 0 ? \
-mkconst(1.0, semantics=instant, units="kbyte / count") : \
delta(disk.dev.total_bytes) / delta(disk.dev.total)
* Expressions may be rescaled using the rescale function that takes
two arguments. The first is an arithmetic expression to be
rescaled, and the second is the desired units after rescaling that
is a string value in the syntax accepted by pmParseUnitsStr(3).
For example:
rescale(network.interface.total.bytes, "Mbytes/hour")
The expression and the desired units must both have the same
dimension, e.g Space=1, Time=-1 and Count=0 in the example above.
* The following unary functions operate on a single performance
metric and return one or more values. For all functions (except
count(), defined() and instant()), the type of the operand metric
must be arithmetic (integer of various sizes and signedness, float
or double).
┌───────────┬───────────────────────────────────────────────┐
│ Function │ Value │
├───────────┼───────────────────────────────────────────────┤
│avg(x) │ A singular instance being the average value │
│ │ across all instances for the metric x. │
├───────────┼───────────────────────────────────────────────┤
│count(x) │ A singular instance being the count of the │
│ │ number of instances for the metric x. As a │
│ │ special case, if fetching the metric x │
│ │ returns an error, then count(x) will be 0. │
├───────────┼───────────────────────────────────────────────┤
│defined(x) │ A boolean value that is true (``1'') if the │
│ │ metric x is defined in the PMNS, else false │
│ │ (``0''). The function is evaluated when a │
│ │ new PMAPI context is created with │
│ │ pmNewContext(3) or re-established with │
│ │ pmReconnectContext(3). So any subsequent │
│ │ changes to the PMNS after the PMAPI context │
│ │ has been established will not change the │
│ │ value of this function in the expression │
│ │ evaluation. │
├───────────┼───────────────────────────────────────────────┤
│delta(x) │ Returns the difference in values for the │
│ │ metric x between one call to pmFetch(3) and │
│ │ the next. There is one value in the result │
│ │ for each instance that appears in both the │
│ │ current and the previous sample. │
├───────────┼───────────────────────────────────────────────┤
│rate(x) │ Returns the difference in values for the │
│ │ metric x between one call to pmFetch(3) and │
│ │ the next divided by the elapsed time between │
│ │ the calls to pmFetch(3). The semantics of │
│ │ the derived metric are based on the semantics │
│ │ of the operand (x) with the dimension in the │
│ │ time domain decreased by one and scaling if │
│ │ required in the time utilization case where │
│ │ the operand is in units of time, and the │
│ │ derived metric is unitless. This mimics the │
│ │ rate conversion applied to counter metrics by │
│ │ tools such as pmval(1), pmie(1) and │
│ │ pmchart(1). There is one value in the result │
│ │ for each instance that appears in both the │
│ │ current and the previous sample. │
├───────────┼───────────────────────────────────────────────┤
│instant(x) │ Returns the current value of the metric x, │
│ │ even it has the semantics of a counter, i.e. │
│ │ PM_SEM_COUNTER. The semantics of the derived │
│ │ metric are based on the semantics of the │
│ │ operand (x); if x has semantics │
│ │ PM_SEM_COUNTER, the semantics of instant(x) │
│ │ is PM_SEM_INSTANT, otherwise the semantics of │
│ │ the derived metric is the same as the │
│ │ semantics of the metric x. │
├───────────┼───────────────────────────────────────────────┤
│max(x) │ A singular instance being the maximum value │
│ │ across all instances for the metric x. │
├───────────┼───────────────────────────────────────────────┤
│min(x) │ A singular instance being the minimum value │
│ │ across all instances for the metric x. │
├───────────┼───────────────────────────────────────────────┤
│sum(x) │ A singular instance being the sum of the │
│ │ values across all instances for the metric x. │
└───────────┴───────────────────────────────────────────────┘
* Parenthesis may be used for explicit grouping.
* Lines beginning with ``#'' are treated as comments and ignored.
* White space is ignored.
There are a number of conversions required to determine the metadata
for a derived metric and to ensure the semantics of the expressions
are sound.
In an arithmetic expression or a relational expression, if the
semantics of both operands is not a counter (i.e. PM_SEM_INSTANT or
PM_SEM_DISCRETE) then the result will have semantics PM_SEM_INSTANT
unless both operands are PM_SEM_DISCRETE in which case the result is
also PM_SEM_DISCRETE.
For an arithmetic expression, the dimension of each operand must be
the same. For a relational expression, the dimension of each operand
must be the same, except that numeric constants (with no dimension)
are allowed, e.g. in the expression network.interface.in.drops > 0 .
To prevent arbitrary and non-sensical combinations some restrictions
apply to expressions that combine metrics with counter semantics to
produce a result with counter semantics. For an arithmetic
expression, if both operands have the semantics of a counter, then
only addition or subraction is allowed, or if the left operand is a
counter and the right operand is not, then only multiplication or
division are allowed, or if the left operand is not a counter and the
right operand is a counter, then only multiplication is allowed.
Because relational expressions use the current value only and produce
a result that is not a counter, either or both operands of a
relational expression may be counters.
The mapping of the pmUnits of the metadata uses the following rules:
* If both operands have a dimension of Count and the scales are not
the same, use the larger scale and convert the values of the
operand with the smaller scale.
* If both operands have a dimension of Time and the scales are not
the same, use the larger scale and convert the values of the
operand with the smaller scale.
* If both operands have a dimension of Space and the scales are not
the same, use the larger scale and convert the values of the
operand with the smaller scale.
* For addition and subtraction all dimensions for each of the
operands and result are identical.
* For multiplication, the dimensions of the result are the sum of the
dimensions of the operands.
* For division, the dimensions of the result are the difference of
the dimensions of the operands.
Scale conversion involves division if the dimension is positive else
multiplication if the dimension is negative. If scale conversion is
applied to either of the operands, the result is promoted to type
PM_TYPE_DOUBLE.
Putting all of this together in an example, consider the derived
metric defined as follows:
x = network.interface.speed - delta(network.interface.in.bytes) /
delta(sample.milliseconds)
The type, dimension and scale settings would propagate up the
expression tree as follows.
┌────────────────────────┬────────┬───────────────┬─────────────────┐
│ Expression │ Type │ Dimension & │ Scale Factor(s) │
│ │ │ Scale │ │
├────────────────────────┼────────┼───────────────┼─────────────────┤
│sample.milliseconds │ DOUBLE │ millisec │ │
│delta(...) │ DOUBLE │ millisec │ │
│network...bytes │ U64 │ byte │ │
│delta(...) │ U64 │ byte │ │
│delta(...) / delta(...) │ DOUBLE │ byte/millisec │ /1048576 and │
│ │ │ │ *1000 │
│network...speed │ FLOAT │ Mbyte/sec │ │
│x │ DOUBLE │ Mbyte/sec │ │
└────────────────────────┴────────┴───────────────┴─────────────────┘
Because semantic checking cannot be done at the time
pmRegisterDerived is called, errors found during semantic checking
(when any subsequent calls to pmNewContext(3) or
pmReconnectContext(3) succeed) are reported using pmprintf(3). These
include:
Error: derived metric <name1>: operand: <name2>: <reason>
There was a problem calling pmLookupName(3) to identify the
operand metric <name2> used in the definition of the derived
metric <name1>.
Error: derived metric <name1>: operand (<name2> [<pmid2>]): <reason>
There was a problem calling pmLookupDesc(3) to identify the
operand metric <name2> with PMID <pmid2> used in the
definition of the derived metric <name1>.
Semantic error: derived metric <name>: <operand> : <operand>
Different <metadata> for ternary operands
For a ternary expression, the ``true'' operand and the
``false'' operand must have exactly the same metadata, so
type, semantics, instance domain, and units (dimension and
scale).
Semantic error: derived metric <name>: <operand> <op> <operand>:
Dimensions are not the same
Operands must have the same units (dimension and scale) for
each of addition, subtraction, the relational operators and
the boolean ``and'' or ``or'' operators.
Semantic error: derived metric <name>: <operand> <op> <operand>:
Illegal operator for counter and non-counter
Only multiplication or division are allowed if the left
operand has the semantics of a counter and the right operand
is not a counter.
Semantic error: derived metric <name>: <operand> <op> <operand>:
Illegal operator for counters
If both operands have the semantics of counter, only addition
or subtraction make sense, so multiplication and division are
not allowed.
Semantic error: derived metric <name>: <operand> <op> <operand>:
Illegal operator for non-counter and counter
Only multiplication is allowed if the right operand has the
semantics of a counter and the left operand is not a counter.
Semantic error: derived metric <metric> <expr> RESCALE <units>:
Incompatible dimensions
The parameters <expr> and <units> to the rescale function must
have the same dimension along the axes of Time, Space and
Count.
Semantic error: derived metric <name>: Incorrect time dimension for
operand
Rate conversion using the rate() function is only possible for
operand metrics with a Time dimension of 0 or 1 (see
pmLookupDesc(3)). If the operand metric's Time dimension is
0, then the derived metrics has a value "per second" (Time
dimension of -1). If the operand metric's Time dimension is
1, then the derived metrics has a value of time utilization
(Time dimension of 0).
Semantic error: derived metric <name>: <function>(<operand>): Non-
arithmetic operand for function
The unary functions are only defined if the operand has
arithmetic type. Similarly the first argument to the rescale
function must be of arithmetic type.
Semantic error: derived metric <name>: <expr> ? ...: Non-arithmetic
operand for ternary guard
The first expression for a ternary operator must have an
arithmetic type.
Semantic error: derived metric <name>: ... - ...: Non-arithmetic
operand for unary negation
Unary negation only makes sense if the following expression
has an arithmetic type.
Semantic error: derived metric <name>: <operand> <op> <operand>: Non-
arithmetic type for <left-or-right> operand
The binary arithmetic operators are only allowed with operands
with an arithmetic type (integer of various sizes and
signedness, float or double).
Semantic error: derived metric <name>: <operand> <op> <operand>: Non-
counter and not dimensionless <left-or-right> operand
For multiplication or division or any of the relational
operators, if one of the operands has the semantics of a
counter and the other has the semantics of a non-counter
(instantaneous or discrete) then the non-counter operand must
have no units (dimension and scale).
Semantic error: derived metric <name>: <expr> ? <expr> : <expr>: Non-
scalar ternary guard with scalar expressions
If the ``true'' and ``false'' operands of a ternary expression
have a scalar value, then the ``guard'' expression must also
have a scalar value.
Semantic error: derived metric <name>: <expr> <op> <expr>: Operands
should have the same instance domain
For all of the binary operators (arithmetic and relational),
if both operands have non-scalar values, then they must be
defined over the same instance domain.
For the binary arithmetic operators, if either operand must be scaled
(e.g. convert bytes to Kbytes) then the result is promoted to
PM_TYPE_DOUBLE. Otherwise the type of the result is determined by
the types of the operands, as per the following table which is
evaluated from top to bottom until a match is found.
┌─────────────────────────┬──────────┬────────────────┐
│ Operand Types │ Operator │ Result Type │
├─────────────────────────┼──────────┼────────────────┤
│either is PM_TYPE_DOUBLE │ any │ PM_TYPE_DOUBLE │
├─────────────────────────┼──────────┼────────────────┤
│any │ division │ PM_TYPE_DOUBLE │
├─────────────────────────┼──────────┼────────────────┤
│either is PM_TYPE_FLOAT │ any │ PM_TYPE_FLOAT │
├─────────────────────────┼──────────┼────────────────┤
│either is PM_TYPE_U64 │ any │ PM_TYPE_U64 │
├─────────────────────────┼──────────┼────────────────┤
│either is PM_TYPE_64 │ any │ PM_TYPE_64 │
├─────────────────────────┼──────────┼────────────────┤
│either is PM_TYPE_U32 │ any │ PM_TYPE_U32 │
├─────────────────────────┼──────────┼────────────────┤
│otherwise (both are │ any │ PM_TYPE_32 │
│PM_TYPE_32) │ │ │
└─────────────────────────┴──────────┴────────────────┘
Derived metrics are not available when using pmFetchArchive(3) as
this routine does not use a target list of PMIDs that could be
remapped (as is done for pmFetch(3)).
There is no pmUnregisterDerived method, so once registered a derived
metric persists for the life of the application.
On success, pmRegisterDerived returns NULL.
If a syntactic error is found at the time of registration, the value
returned by pmRegisterDerived is a pointer into expr indicating where
the error was found. To identify what the error was, the application
should call pmDerivedErrStr(3) to retrieve the corresponding parser
error message.
pmRegisterDerivedMetric returns 0 and errmsg is undefined if the
parsing is successful.
If the given expr does not conform to the required syntax
pmRegisterDerivedMetric returns -1 and a dynamically allocated error
message string in errmsg. The error message is terminated with a
newline and includes both the input name and expr, along with an
indicator of the position at which the error was detected. e.g.
Error: pmRegisterDerivedMetric("my.disk.rates", ...) syntax
error
4rat(disk.dev.read)
^
The position indicator line may be followed by an additional
diagnostic line describing the nature of the error, when available.
In the case of an error, the caller is responsible for calling
free(3) to release the space allocated for errmsg.
PCPIntro(1), free(3), PMAPI(3), pmDerivedErrStr(3), pmFetch(3),
pmLoadDerivedConfig(3), pmNewContext(3), pmprintf(3) and
pmReconnectContext(3).
This page is part of the PCP (Performance Co-Pilot) project.
Information about the project can be found at ⟨http://www.pcp.io/⟩.
If you have a bug report for this manual page, send it to
pcp@groups.io. This page was obtained from the project's upstream
Git repository ⟨https://github.com/performancecopilot/pcp.git⟩ on
2018-02-02. (At that time, the date of the most recent commit that
was found in the repository was 2018-02-02.) If you discover any
rendering problems in this HTML version of the page, or you believe
there is a better or more up-to-date source for the page, or you have
corrections or improvements to the information in this COLOPHON
(which is not part of the original manual page), send a mail to
man-pages@man7.org
Performance Co-Pilot PMREGISTERDERIVED(3)
Pages that refer to this page: pcp2elasticsearch(1), pcp2graphite(1), pcp2influxdb(1), pcp2json(1), pcp2xlsx(1), pcp2xml(1), pcp2zabbix(1), pmrep(1), pmderivederrstr(3), pmfetchgroup(3), pmloadderivedconfig(3), pmreconnectcontext(3), pmrep.conf(5)
|
HTML rendering created 2018-02-02 by Michael Kerrisk, author of The Linux Programming Interface, maintainer of the Linux man-pages project. For details of in-depth Linux/UNIX system programming training courses that I teach, look here. Hosting by jambit GmbH.
|
|