mirror of
https://github.com/asterisk/asterisk.git
synced 2026-04-23 15:17:10 +00:00
4871413877
There is a LOT of work in this commit but the TL;DR is that it takes CEL processing from using 38% of the CPU instructions used by a call, which is more than that used by the call processing itself, down to less than 10% of the instructions. So here's the deal... cdr_custom, cdr_sqlite3_custom, cel_custom and cel_sqlite3_custom all shared one ugly trait...they all used ast_str_substitute_variables() or pbx_substitute_variables_helper() to resolve the dialplan functions used in their config files. Not only are they both extraordinarily expensive, they both require a dummy channel to be allocated and destroyed for each record written. For CDRs, that's not too bad because we only write one CDR per call. For CELs however, it's a disaster. As far as source code goes, the modules basically all did the same thing. Unfortunately, they did it badly. The config files simply contained long opaque strings which were intepreted by ast_str_substitute_variables() or pbx_substitute_variables_helper(), the very functions that ate all the instructions. This meant introducing a new "advanced" config format much like the advanced manager event filtering added to manager.conf in 2024. Fortunately however, if the legacy config was recognizable, we were able to parse it as an advanced config and gain the benefit. If not, then it goes the legacy, and very expensive, route. Given the commonality among the modules, instead of making the same improvements to 4 modules then trying to maintain them over time, a single module "res_cdrel_custom" was created that contains all of the common code. A few bonuses became possible in the process... * The cdr_custom and cel_custom modules now support JSON formatted output. * The cdr_sqlite_custom and cel_sqlite3_custom modules no longer have to share an Sqlite3 database. Summary of changes: A new module "res/res_cdrel_custom.c" has been created and the existing cdr_custom, cdr_sqlite3_custom, cel_custom and cel_sqlite3_custom modules are now just stubs that call the code in res_cdrel_custom. res_cdrel_custom contains: * A common configuration facility. * Getters for both CDR and CEL fields that share the same abstraction. * Formatters for all data types found in the ast_cdr and ast_event structures that share the same abstraction. * Common writers for the text file and database backends that, you guessed it, share the same abstraction. The result is that while there is certainly a net increase in the number of lines in the code base, most of it is in the configuration handling at load-time. The run-time instruction path length is now significanty shorter. ``` Scenario Instructions Latency ===================================================== CEL pre changes 38.49% 37.51% CEL Advanced 9.68% 6.06% CEL Legacy (auto-conv to adv) 9.95% 6.13% CEL Sqlite3 pre changes 39.41% 39.90% CEL Sqlite3 Advanced 25.68% 24.24% CEL Sqlite3 Legacy (auto conv) 25.88% 24.53% CDR pre changes 4.79% 2.95% CDR Advanced 0.79% 0.47% CDR Legacy (auto conv to adv) 0.86% 0.51% CDR Sqlite3 pre changes 4.47% 2.89% CEL Sqlite3 Advanced 2.16% 1.29% CEL Sqlite3 Legacy (auto conv) 2.19% 1.30% ``` Notes: * We only write one CDR per call but every little bit helps. * Sqlite3 still takes a fair amount of resources but the new config makes a decent improvement. * Legacy configs that we can't auto convert will still take the "pre changes" path. If you're interested in more implementation details, see the comments at the top of the res_cdrel_custom.c file. One minor fix to CEL is also included...Although TenantID was added to the ast_event structure, it was always rendered as an empty string. It's now properly rendered. UserNote: Significant performance improvements have been made to the cdr_custom, cdr_sqlite3_custom, cel_custom and cel_sqlite3_custom modules. See the new sample config files for those modules to see how to benefit from them.
117 lines
6.2 KiB
Plaintext
117 lines
6.2 KiB
Plaintext
;
|
|
; Asterisk Channel Event Logging (CEL) - Custom Sqlite3 Backend
|
|
;
|
|
|
|
; This is the configuration file for the customizable Sqlite3 backend for CEL
|
|
; logging.
|
|
;
|
|
; Legacy vs Advanced Mappings
|
|
;
|
|
; Legacy mappings are those that are defined using dialplan functions like
|
|
; CALLERID and CHANNEL and require a VERY expensive function replacement
|
|
; process at runtime for every record output.
|
|
;
|
|
; Advanced mappings are those that are defined by a simple list of field names
|
|
; and uses significantly less resources at runtime.
|
|
;
|
|
;
|
|
; Each category in this file represents a separate Sqlite3 database file.
|
|
; A filename that starts with a forward-slash '/' will be treated as an absolute
|
|
; path name. If it doesn't, it must be a file name with no directory separators
|
|
; which will be placed in the /var/log/asterisk directory. If the database
|
|
; file doesn't already exist, it will be created.
|
|
;
|
|
; Previous versions of Asterisk limited the output to a single database file
|
|
; named "master" which was shared with CDR Sqlite3 logging. That is no longer
|
|
; the case.
|
|
;
|
|
;
|
|
; There are several "special" variables created by this module that can be used
|
|
; in a mapping, both legacy and advanced:
|
|
;
|
|
; eventtype - The name of the CEL event.
|
|
; eventtime - The timestamp of the CEL event.
|
|
; eventenum - Like eventtype but is "USER_DEFINED" for a user defined event.
|
|
; userdeftype - User defined event type name from CELGenUserEvent().
|
|
; eventextra - Extra data included with this CEL event, typically along with
|
|
; an event of type USER_DEFINED from CELGenUserEvent().
|
|
; BRIDGEPEER - Bridged peer channel name at the time of the CEL event.
|
|
; CHANNEL(peer) could also be used.
|
|
;
|
|
; Legacy Mappings
|
|
;
|
|
; Within a legacy mapping, use the CALLERID() and CHANNEL() functions or
|
|
; the special variables above to retrieve values from the CEL event.
|
|
;
|
|
; NOTE: If your legacy mapping uses only those two functions and the special
|
|
; variables, the module will attempt to strip the functions and create a much
|
|
; faster advanced mapping for it. However, we urge you to create a real advanced
|
|
; mapping and not rely on this process. If the mapping contains something not
|
|
; recognized it will go the slower legacy route.
|
|
;
|
|
;
|
|
;[cel_master]
|
|
;table = cel ; The name of the table in the database into which records
|
|
; are to be written. If the table doesn't already exist,
|
|
; it will be created.
|
|
|
|
;columns = eventtype, eventtime, cidname, cidnum, cidani, cidrdnis, ciddnid, context, exten, channame, appname, appdata, amaflags, accountcode, uniqueid, userfield, peer, userdeftype, eventextra
|
|
; The column names to receive the fields. If the table doesn't already exist,
|
|
; it will be created with these columns. If the table does exist, this list
|
|
; MUST match the existing columns or the config will fail to load.
|
|
; The column names do NOT have to match the field names however.
|
|
|
|
values = '${eventtype}','${eventtime}','${CALLERID(name)}','${CALLERID(num)}','${CALLERID(ANI)}','${CALLERID(RDNIS)}','${CALLERID(DNID)}','${CHANNEL(context)}','${CHANNEL(exten)}','${CHANNEL(channame)}','${CHANNEL(appname)}','${CHANNEL(appdata)}','${CHANNEL(amaflags)}','${CHANNEL(accountcode)}','${CHANNEL(uniqueid)}','${CHANNEL(userfield)}','${BRIDGEPEER}','${userdeftype}','${eventextra}'
|
|
; The list of fields to write into the columns.
|
|
; Each field MUST be enclosed in single-quotes and the fields separated
|
|
; by commas. Additionally, the number of fields specified MUST match the
|
|
; number of columns or the config will fail to load.
|
|
|
|
;busy_timeout = 1000 ; The number of milliseconds to wait for a database operation
|
|
; to complete before an error is returned.
|
|
|
|
;
|
|
; Advanced Mappings
|
|
;
|
|
;[cdr_advanced]
|
|
;table = cel ; The name of the table in the database into which records
|
|
; are to be written. If the table doesn't already exist,
|
|
; it will be created.
|
|
|
|
;columns = eventtype, literal, eventtime, cidname, cidnum, cidani, cidrdnis, ciddnid, context, exten, channame, appname, appdata, amaflags, accountcode, uniqueid, userfield, peer, userdeftype, eventextra
|
|
; The column names to receive the fields. If the table doesn't already exist,
|
|
; it will be created with these columns. If the table does exist, this list
|
|
; MUST match the existing columns or the config will fail to load.
|
|
; The column names do NOT have to match the field names however.
|
|
|
|
;fields = eventtype, "some literal", eventtime, name, num, ani, rdnis, dnid, context, exten, channame, appname, appdata, amaflags, accountcode, uniqueid, userfield, peer, userdeftype, eventextra
|
|
; The "fields" parameter differentiates this mapping as an Advanced one
|
|
; as opposed to "values" used above.
|
|
;
|
|
; This is the list of fields to include in the record. The field names are the
|
|
; same as in the legacy mapping but without any enclosing dialplan functions or
|
|
; quotes. You can specify literals to be placed in the output record by
|
|
; double-quoting them. There is also some special notation available in the
|
|
; form of "qualifiers". A qualifier is a list of tags, separated by the '^'
|
|
; character and placed directly after the field name and enclosed in parentheses.
|
|
;
|
|
; The default output format for the "EventTime" timestamp field is the "%Y-%m-%d %T"
|
|
; strftime string format however you can also format the field as an int64 or a
|
|
; float: `eventtime(int64)` or `eventtime(float)`.
|
|
;
|
|
; Unlike CDRs, the "amaflags" field is output as its numerical value by default
|
|
; for historical reasons. You can output it as its friendly string with
|
|
; `amaflags(amaflags)`. This will print "DOCUMENTATION" instead of "3" for instance.
|
|
;
|
|
; If you need to combine flags, use the caret '^' symbol: `eventtime(int64^noquote)`
|
|
;
|
|
; Final notes about "fields":
|
|
; Field names and qualifiers aren't case sensitive.
|
|
; You MUST use commas to separate the fields.
|
|
; You MUST use double-quotes to indicate literal fields.
|
|
; Whitespace in "fields" is ignored except in literals.
|
|
|
|
;busy_timeout = 1000 ; The number of milliseconds to wait for a database operation
|
|
; to complete before an error is returned.
|
|
|