188.8.131.52 Event Buffer Reporting in the Cluster Log
NDB uses one or more memory buffers for events received from the data nodes. There is one such buffer for each
Ndb object subscribing to table events, which means that there are usually two buffers for each mysqld performing binary logging (one buffer for schema events, and one for data events). Each buffer contains epochs made up of events. These events consist of operation types (insert, update, delete) and row data (before and after images plus metadata).
NDB generates messages in the cluster log to describe the state of these buffers. Although these reports appear in the cluster log, they refer to buffers on API nodes (unlike most other cluster log messages, which are generated by data nodes). These messages and the data structures underlying them were changed significantly in NDB 7.5.1, with the addition of the
NDB_LE_EventBufferStatus2 event type and the
ndb_logevent_EventBufferStatus2 data structure (see The Ndb_logevent_type Type ). The remainder of this discussion focuses on the implementation based on
Event buffer logging reports in the cluster log use the format shown here:
Node node_id: Event buffer status (object_id): used=bytes_used (percent_used% of alloc) alloc=bytes_allocated (percent_alloc% of max) max=bytes_available latest_consumed_epoch=latest_consumed_epoch latest_buffered_epoch=latest_buffered_epoch report_reason=report_reason
The fields making up this report are listed here, with descriptions:
node_id: ID of the node where the report originated.
object_id: ID of the
Ndbobject where the report originated.
bytes_used: Number of bytes used by the buffer.
percent_used: Percentage of allocated bytes used.
bytes_allocated: Number of bytes allocated to this buffer.
percent_alloc: Percentage of available bytes used; not printed if
ndb_eventbuffer_max_allocis equal to 0 (unlimited).
bytes_available: Number of bytes available; this is 0 if
ndb_eventbuffer_max_allocis 0 (unlimited).
latest_consumed_epoch: The epoch most recently consumed to completion. (In NDB API applications, this is done by calling
latest_buffered_epoch: The epoch most recently buffered (completely) in the event buffer.
report_reason: The reason for making the report. Possible reasons are shown later in this section.
latest_buffered_epoch fields correspond, respectively, to the
latest_gci fields of the old-style event buffer logging messages used prior to NDB 7.5.1.
Possible reasons for reporting are described in the following list:
ENOUGH_FREE_EVENTBUFFER: The event buffer has sufficient space.
LOW_FREE_EVENTBUFFER: The event buffer is running low on free space.
The threshold free percentage level triggering these reports can be adjusted by setting the
BUFFERED_EPOCHS_OVER_THRESHOLD: Whether the number of buffered epochs has exceeded the configured threshold. This number is the difference between the latest epoch that has been received in its entirety and the epoch that has most recently been consumed (in NDB API applications, this is done by calling
nextEvent2()). The report is generated every second until the number of buffered epochs goes below the threshold, which can be adjusted by setting the
ndb_report_thresh_binlog_epoch_slipserver variable. You can also adjust the threshold in NDB API applications by calling
PARTIALLY_DISCARDING: Event buffer memory is exhausted—that is, 100% of
ndb_eventbuffer_max_allochas been used. Any partially buffered epoch is buffered to completion even is usage exceeds 100%, but any new epochs received are discarded. This means that a gap has occurred in the event stream.
COMPLETELY_DISCARDING: No epochs are buffered.
PARTIALLY_BUFFERING: The buffer free percentage following the gap has risen to the threshold, which can be set in the mysql client using the
ndb_eventbuffer_free_percentserver system variable or in NDB API applications by calling
set_eventbuffer_free_percent(). New epochs are buffered. Epochs that could not be completed due to the gap are discarded.
COMPLETELY_BUFFERING: All epochs received are being buffered, which means that there is sufficient event buffer memory. The gap in the event stream has been closed.