There is a LOT of work in this commit but the TL;DR is that it takes
CEL processing from using 38% of the CPU instructions used by a call,
which is more than that used by the call processing itself, down to less
than 10% of the instructions.
So here's the deal... cdr_custom, cdr_sqlite3_custom, cel_custom
and cel_sqlite3_custom all shared one ugly trait...they all used
ast_str_substitute_variables() or pbx_substitute_variables_helper()
to resolve the dialplan functions used in their config files. Not only
are they both extraordinarily expensive, they both require a dummy
channel to be allocated and destroyed for each record written. For CDRs,
that's not too bad because we only write one CDR per call. For CELs however,
it's a disaster.
As far as source code goes, the modules basically all did the same thing.
Unfortunately, they did it badly. The config files simply contained long
opaque strings which were intepreted by ast_str_substitute_variables() or
pbx_substitute_variables_helper(), the very functions that ate all the
instructions. This meant introducing a new "advanced" config format much
like the advanced manager event filtering added to manager.conf in 2024.
Fortunately however, if the legacy config was recognizable, we were able to
parse it as an advanced config and gain the benefit. If not, then it
goes the legacy, and very expensive, route.
Given the commonality among the modules, instead of making the same
improvements to 4 modules then trying to maintain them over time, a single
module "res_cdrel_custom" was created that contains all of the common code.
A few bonuses became possible in the process...
* The cdr_custom and cel_custom modules now support JSON formatted output.
* The cdr_sqlite_custom and cel_sqlite3_custom modules no longer have
to share an Sqlite3 database.
Summary of changes:
A new module "res/res_cdrel_custom.c" has been created and the existing
cdr_custom, cdr_sqlite3_custom, cel_custom and cel_sqlite3_custom modules
are now just stubs that call the code in res_cdrel_custom.
res_cdrel_custom contains:
* A common configuration facility.
* Getters for both CDR and CEL fields that share the same abstraction.
* Formatters for all data types found in the ast_cdr and ast_event
structures that share the same abstraction.
* Common writers for the text file and database backends that, you guessed it,
share the same abstraction.
The result is that while there is certainly a net increase in the number of
lines in the code base, most of it is in the configuration handling at
load-time. The run-time instruction path length is now significanty shorter.
```
Scenario Instructions Latency
=====================================================
CEL pre changes 38.49% 37.51%
CEL Advanced 9.68% 6.06%
CEL Legacy (auto-conv to adv) 9.95% 6.13%
CEL Sqlite3 pre changes 39.41% 39.90%
CEL Sqlite3 Advanced 25.68% 24.24%
CEL Sqlite3 Legacy (auto conv) 25.88% 24.53%
CDR pre changes 4.79% 2.95%
CDR Advanced 0.79% 0.47%
CDR Legacy (auto conv to adv) 0.86% 0.51%
CDR Sqlite3 pre changes 4.47% 2.89%
CEL Sqlite3 Advanced 2.16% 1.29%
CEL Sqlite3 Legacy (auto conv) 2.19% 1.30%
```
Notes:
* We only write one CDR per call but every little bit helps.
* Sqlite3 still takes a fair amount of resources but the new config
makes a decent improvement.
* Legacy configs that we can't auto convert will still take the
"pre changes" path.
If you're interested in more implementation details, see the comments
at the top of the res_cdrel_custom.c file.
One minor fix to CEL is also included...Although TenantID was added to the
ast_event structure, it was always rendered as an empty string. It's now
properly rendered.
UserNote: Significant performance improvements have been made to the
cdr_custom, cdr_sqlite3_custom, cel_custom and cel_sqlite3_custom modules.
See the new sample config files for those modules to see how to benefit
from them.
* acl (named_acl.c)
* cdr
* cel
* ccss
* dnsmgr
* dsp
* enum
* extconfig (config.c)
* features
* http
* indications
* logger
* manager
* plc
* sounds
* udptl
These modules are now loaded at appropriate time by the module loader.
Unlike loadable modules these use AST_MODULE_LOAD_FAILURE on error so
the module loader will abort startup on failure of these modules.
Some of these modules are still initialized or shutdown from outside the
module loader. logger.c is initialized very early and shutdown very
late, manager.c is initialized by the module loader but is shutdown by
the Asterisk core (too much uses it without holding references).
Change-Id: I371a9a45064f20026c492623ea8062d02a1ab97f
ASTERISK_REGISTER_FILE no longer has any purpose so this commit removes
all traces of it.
Previously exported symbols removed:
* __ast_register_file
* __ast_unregister_file
* ast_complete_source_filename
This also removes the mtx_prof static variable that was declared when
MTX_PROFILE was enabled. This variable was only used in lock.c so it
is now initialized in that file only.
ASTERISK-26480 #close
Change-Id: I1074af07d71f9e159c48ef36631aa432c86f9966
Git does not support the ability to replace a token with a version
string during check-in. While it does have support for replacing a
token on clone, this is somewhat sub-optimal: the token is replaced
with the object hash, which is not particularly easy for human
consumption. What's more, in practice, the source file version was often
not terribly useful. Generally, when triaging bugs, the overall version
of Asterisk is far more useful than an individual SVN version of a file. As a
result, this patch removes Asterisk's support for showing source file
versions.
Specifically, it does the following:
* Rename ASTERISK_FILE_VERSION macro to ASTERISK_REGISTER_FILE, and
remove passing the version in with the macro. Other facilities
than 'core show file version' make use of the file names, such as
setting a debug level only on a specific file. As such, the act of
registering source files with the Asterisk core still has use. The
macro rename now reflects the new macro purpose.
* main/asterisk:
- Refactor the file_version structure to reflect that it no longer
tracks a version field.
- Remove the "core show file version" CLI command. Without the file
version, it is no longer useful.
- Remove the ast_file_version_find function. The file version is no
longer tracked.
- Rename ast_register_file_version/ast_unregister_file_version to
ast_register_file/ast_unregister_file, respectively.
* main/manager: Remove value from the Version key of the ModuleCheck
Action. The actual key itself has not been removed, as doing so would
absolutely constitute a backwards incompatible change. However, since
the file version is no longer tracked, there is no need to attempt to
include it in the Version key.
* UPGRADE: Add notes for:
- Modification to the ModuleCheck AMI Action
- Removal of the "core show file version" CLI command
Change-Id: I6cf0ff280e1668bf4957dc21f32a5ff43444a40e
This patch addresses some aesthetic issues in Asterisk. These are all just
minor tweaks to improve the look of the CLI when used in a variety of
settings. Specifically:
* A number of chatty verbose messages were removed or demoted to DEBUG
messages. Verbose messages with a verbosity level of 5 or higher were -
if kept as verbose messages - demoted to level 4. Several messages
that were emitted at verbose level 3 were demoted to 4, as announcement
of dialplan applications being executed occur at level 3 (and so the
effects of those applications should generally be less).
* Some verbose messages that only appear when their respective 'debug'
options are enabled were bumped up to always be displayed.
* Prefix/timestamping of verbose messages were moved to the verboser
handlers. This was done to prevent duplication of prefixes when the
timestamp option (-T) is used with the CLI.
* Verbose magic is removed from messages before being emitted to
non-verboser handlers. This prevents the magic in multi-line verbose
messages (such as SIP debug traces or the output of DumpChan) from
being written to files.
* _Slightly_ better support for the "light background" option (-W) was
added. This includes using ast_term_quit in the output of XML
documentation help, as well as changing the "Asterisk Ready" prompt to
bright green on the default background (which stands a better chance of
being displayed properly than bright white).
Review: https://reviewboard.asterisk.org/r/3547/
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@414798 65c4cc65-6c06-0410-ace0-fbb531ad65f3
https://origsvn.digium.com/svn/asterisk/branches/10
................
r337974 | rmudgett | 2011-09-26 14:35:23 -0500 (Mon, 26 Sep 2011) | 37 lines
Merged revisions 337973 via svnmerge from
https://origsvn.digium.com/svn/asterisk/branches/1.8
........
r337973 | rmudgett | 2011-09-26 14:30:39 -0500 (Mon, 26 Sep 2011) | 30 lines
Fix deadlock when using dummy channels.
Dummy channels created by ast_dummy_channel_alloc() should be destoyed by
ast_channel_unref(). Using ast_channel_release() needlessly grabs the
channel container lock and can cause a deadlock as a result.
* Analyzed use of ast_dummy_channel_alloc() and made use
ast_channel_unref() when done with the dummy channel. (Primary reason for
the reported deadlock.)
* Made app_dial.c:dial_exec_full() not call ast_call() holding any channel
locks. Chan_local could not perform deadlock avoidance correctly.
(Potential deadlock exposed by this issue. Secondary reason for the
reported deadlock since the held lock was part of the deadlock chain.)
* Fixed some uses of ast_dummy_channel_alloc() not checking the returned
channel pointer for failure.
* Fixed some potential chan=NULL pointer usage in func_odbc.c. Protected
by testing the bogus_chan value.
* Fixed needlessly clearing a 1024 char auto array when setting the first
char to zero is enough in manager.c:action_getvar().
(closes issue ASTERISK-18613)
Reported by: Thomas Arimont
Patches:
jira_asterisk_18613_v1.8.patch (license #5621) patch uploaded by rmudgett
Tested by: Thomas Arimont
........
................
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@337975 65c4cc65-6c06-0410-ace0-fbb531ad65f3
CEL is the new system for logging channel events. This was inspired after
facing many problems trying to represent what is possible to happen to a call
in Asterisk using CDR records. For more information on CEL, see the built in
HTML or PDF documentation generated from the files in doc/tex/.
Many thanks to Steve Murphy (murf) and Brian Degenhardt (bmd) for their hard
work developing this code. Also, thanks to Matt Nicholson (mnicholson) and
Sean Bright (seanbright) for their assistance in the final push to get this
code ready for Asterisk trunk.
Review: https://reviewboard.asterisk.org/r/239/
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@203638 65c4cc65-6c06-0410-ace0-fbb531ad65f3
This patch adds 'const' tags to a number of Asterisk APIs where they are appropriate (where the API already demanded that the function argument not be modified, but the compiler was not informed of that fact). The list includes:
- CLI command handlers
- CLI command handler arguments
- AGI command handlers
- AGI command handler arguments
- Dialplan application handler arguments
- Speech engine API function arguments
In addition, various file-scope and function-scope constant arrays got 'const' and/or 'static' qualifiers where they were missing.
Review: https://reviewboard.asterisk.org/r/251/
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@196072 65c4cc65-6c06-0410-ace0-fbb531ad65f3
actual problems, per se. I also added format attributes to any printf wrapper functions I found that didn't have them. -Wsecurity and -Wmissing-format-attribute added to --enable-dev-mode.
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@109447 65c4cc65-6c06-0410-ace0-fbb531ad65f3
build times - tested, there is no measureable difference before and
after this commit.
In this change:
use asterisk/compat.h to include a small set of system headers:
inttypes.h, unistd.h, stddef.h, stddint.h, sys/types.h, stdarg.h,
stdlib.h, alloca.h, stdio.h
Where available, the inclusion is conditional on HAVE_FOO_H as determined
by autoconf.
Normally, source files should not include any of the above system headers,
and instead use either "asterisk.h" or "asterisk/compat.h" which does it
better.
For the time being I have left alone second-level directories
(main/db1-ast, etc.).
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@89333 65c4cc65-6c06-0410-ace0-fbb531ad65f3
* Add new module, cdr_sqlite3_custom which allows logging custom CDRs into a
SQLite3 database. (issue #7149, alerios)
* Add new module, res_config_sqlite, which adds realtime database configuration
support for SQLite version 2. I decided that this was ok since we didn't have
any realtime support for version 3. If someone ports this to version 3, then
version 2 support can be removed or marked deprecated.
(issue #7790, rbarun_proformatique)
* Mark cdr_sqlite as deprecated in favor of cdr_sqlite3_custom.
Also, note that there were other modules on the bug tracker that did not make
the cut because they provided some duplicated functionality. Those are:
* cdr_sqlite3 (issue #6754, moy)
* cdr_sqlite3 (issue #8694, bsd)
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@58866 65c4cc65-6c06-0410-ace0-fbb531ad65f3
