ULog 文件格式
ULog is the file format used for logging system data. The format is self-describing, i.e. it contains the format and message types that are logged.
It can be used for logging device inputs (sensors, etc.), internal states (cpu load, attitude, etc.) and printf log messages.
It can be used for logging device inputs (sensors, etc.), internal states (cpu load, attitude, etc.) and printf
log messages.
这种格式对所有的二进制类型采用小端模式。
数据类型
使用以下二进制类型。 The following binary types are used. They all correspond to the types in C:
类型 | 大小(以字节为单位) |
---|---|
int8_t, uint8_t | 1 |
int16_t, uint16_t | 2 |
int32_t, uint32_t | 4 |
int64_t, uint64_t | 8 |
float | 4 |
double | 8 |
bool, char | 1 |
此外,所有的类型还可以作为数组使用,比如 float[5]
。 Additionally all can be used as an array, eg. float[5]
. In general all strings (char[length]
) do not contain a '\0'
at the end. String comparisons are case sensitive. 字符串比较区分大小写。
文件结构
该文件由三个部分组成:
头部分
头是一个固定大小的部分,具有以下格式(16个字节):
Version 是文件的格式的版本,目前是 1。 Version is the file format version, currently 1. Timestamp is a uint64_t
integer, denotes the start of the logging in microseconds.
定义部分
可变长度部分,包含版本信息、格式定义和 (初始) 参数值。
The Definitions and Data sections consist of a stream of messages. Each starts with this header: 每个数据流包含此标头:
msg_size
is the size of the message in bytes without the header (hdr_size
= 3 bytes). msg_type
defines the content and is one of the following: msg_type
定义内容类型,是以下的一种:
'B' :标记 bitset 报文。
这条消息必须是头后面的第一条消息,这样才有固定的常数偏移量。
compat_flags
: compatible flag bits.compat_flags[0]
, bit 0, DEFAULT_PARAMETERS: if set, the log contains parameter defaults (message 'Q').
The rest of the bits is currently not defined and all must be set to 0. These bits can be used for future ULog changes that are compatible with existing parsers. It means parsers can just ignore the bits if one of the unknown bits is set.
incompat_flags
: 不兼容的标志位。incompat_flags
: incompatible flag bits. The LSB bit of index 0 is set to one if the log contains appended data and at lease one of theappended_offsets
is non-zero. All other bits are undefined und must be set to 0. If a parser finds one of these bits set, it must refuse to parse the log. This can be used to introduce breaking changes that existing parsers cannot handle. 其他位都是未定义的,必须将设置为 0。 如果解析器发现这些位置 1,它必须拒绝解析日志。 这可用于引入现有解析器无法处理的重大更改。appended_offsets
: File offsets (0-based) for appended data. If no data is appended, all offsets must be zero. This can be used to reliably append data for logs that may stop in the middle of a message. A process appending data should do: 如果没有附加数据,则所有偏移量必须为零。 这可以用于消息中途暂停的情况下可靠的添加数据。附加数据的过程应该做到:
置位相关的
incompat_flags
位,设置
append_offsets
的第一个元素为日志文件相对于 0 的长度,然后为数据部分添加有效的任何类型的消息。
It is possible that there are more fields appended at the end of this message in future ULog specifications. This means a parser must not assume a fixed length of this message. If the message is longer than expected (currently 40 bytes), the exceeding bytes must just be ignored. 这意味着解析器必须不能假定此消息的长度是固定的。 如果消息比预期的长(当前为 40 字节),则必须忽略超过的字节。
'F': 可以在另一个定义中作为嵌套类型记录或使用的单个 (组合) 类型的格式定义。
};
key
is a plain string, as in the format message (can also be a custom type), but consists of only a single field without ending ;
, eg. float[3] myvalues
. value
contains the data as described by key
. float[3] myvalues
. value
包含 key
所描述的字段
Note that an information message with a certain key must occur at most once in the entire log. Parsers can store information messages as a dictionary. 解析器可以将报文信息存储为字典。
预定义的信息报文有:
键 | 描述 | 示例值 |
---|---|---|
char[value_len] sys_name | 系统名称 | "PX4" |
char[value_len] ver_hw | 硬件版本 (主板) | "PX4FMU_V4" |
char[value_len] ver_hw_subtype | 主办子版本 (变化的) | "V2" |
char[value_len] ver_sw | 软件版本 (git 标签) | "7f65e01" |
char[value_len] ver_sw_branch | git branch | "master" |
uint32_t ver_sw_release | 软件版本 (见下文) | 0x010401ff |
char[value_len] sys_os_name | 操作系统名称 | "Linux" |
char[value_len] sys_os_ver | 操作系统版本 (git 标签) | "9f82919" |
uint32_t ver_os_release | 操作系统版本 (见下文) | 0x010401ff |
char[value_len] sys_toolchain | 工具链名称 | "GNU GCC" |
char[value_len] sys_toolchain_ver | 工具链版本 | "6.2.1" |
char[value_len] sys_mcu | 芯片名称和修订 | "STM32F42x, rev A" |
char[value_len] sys_uuid | Unique identifier for vehicle (eg. MCU ID) | "392a93e32fa3"... |
char[value_len] log_type | Type of the log (full log if not specified) | "mission" |
char[value_len] replay | 重播日志的文件名如果处于重播模式 | "log001.ulg" |
int32_t time_ref_utc | UTC 时间的秒偏移量 | -3600 |
The format of ver_sw_release
and ver_os_release
is: 0xAABBCCTT, where AA is major, BB is minor, CC is patch and TT is the type. Type is defined as following: >= 0
: development, >= 64
: alpha version, >= 128
: beta version, >= 192
: RC version, == 255
: release version. So for example 0x010402ff translates into the release version v1.4.2. Type is defined as following: >= 0
: development, >= 64
: alpha version, >= 128
: beta version, >= 192
: RC version, == 255
: release version. 所以例如 0x010402ff 转换过来是 v1.4.2 的 release 版本。
This message can also be used in the Data section (this is however the preferred section).
'M':多报文信息。
The same as the information message, except that there can be multiple messages with the same key (parsers store them as a list). The
is_continued
can be used for split-up messages: if set to 1, it is part of the previous message with the same key. Parsers can store all information multi messages as a 2D list, using the same order as the messages occur in the log.is_continued
可以用于分割报文:如果置 1,则它是具有相同键的前一条报文的一部分。 解析器可以将所有多报文信息存储为一个 2D 列表,使用与日志中报文相同的顺序。'P':报文参数。 格式与
message_info_s
相同。 'P': parameter message. Same format asmessage_info_s
. If a parameter dynamically changes during runtime, this message can also be used in the Data section. The data type is restricted to:int32_t
,float
. 数据类型限制为:int32_t
,float
。'Q': parameter default message.
default_types
is a bitfield and defines to which group(s) the value belongs to. At least one bit must be set:1<<0
: system wide default1<<1
: default for the current configuration (e.g. an airframe)
A log may not contain default values for all parameters. In those cases the default is equal to the parameter value, and different default types are treated independently. This message can also be used in the Data section. The data type is restricted to:
int32_t
,float
.
这部分在第一个 message_add_logged_s
或者 message_logging_s
开始之前结束 (以先出现的消息为准) 。
数据部分
以下消息属于本部分:
'A': subscribe a message by name and give it an id that is used in
message_data_s
. This must come before the first correspondingmessage_data_s
. 这必须在第一个对应的message_data_s
之前。multi_id
: the same message format can have multiple instances, for example if the system has two sensors of the same type. The default and first instance must be 0.msg_id
: unique id to matchmessage_data_s
data. The first use must set this to 0, then increase it. The samemsg_id
must not be used twice for different subscriptions, not even after unsubscribing.message_name
: message name to subscribe to. Must match one of themessage_format_s
definitions. 默认值以及第一个实例一定是0.msg_id
:匹配message_data_s
数据的惟一 id。 第一次使用一定要设置为 0,然后递增。 相同的msg_id
不能用于两次不同的订阅,甚至在取消订阅后也不行。msg_name
:订阅的消息名称。 必须匹配其中一个message_format_s
的定义。'R':取消订阅一条消息,以标记它将不再被记录 (当前未使用)。
'D':包含日志数据。
msg_id
:由message_add_logged_s
报文定义。msg_id
: as defined by amessage_add_logged_s
message.data
contains the logged binary message as defined bymessage_format_s
. See above for special treatment of padding fields. 有关填充字段的特殊处理,请参见上文。'L':字符串日志报文,比如打印输出。
timestamp
: 以微秒为单位,log_level
: 和 Linux 内核一样。
名称 | 对应值 | 含义 |
---|---|---|
EMERG | '0' | 系统无法使用 |
ALERT | '1' | 操作必须立即执行 |
CRIT | '2' | 紧急情况 |
ERR | '3' | 错误情况 |
WARNING | '4' | 警告情况 |
NOTICE | '5' | 正常但重要的情况 |
INFO | '6' | 信息 |
DEBUG | '7' | 调试级别的消息 |
'C': Tagged Logged string message
tag
: id representing source of logged message string. It could represent a process, thread or a class depending upon the system architecture. For example, a reference implementation for an onboard computer running multiple processes to control different payloads, external disks, serial devices etc can encode these process identifiers using auint16_t enum
into the tag attribute ofmessage_logging_tagged_s
struct as follows:timestamp
: every logged message (message_add_logged_s
) must include a timestamp field (does not need to be the first field). Its type can be:uint64_t
(currently the only one used),uint32_t
,uint16_t
oruint8_t
. The unit is always microseconds, except foruint8_t
it's milliseconds. A log writer must make sure to log messages often enough to be able to detect wrap-arounds and a log reader must handle wrap-arounds (and take into account dropouts). The timestamp must always be monotonic increasing for a message serie with the samemsg_id
.
参数名 | 对应值 | 含义 |
---|---|---|
EMERG | '0' | 系统无法使用 |
ALERT | '1' | 操作必须立即执行 |
CRIT | '2' | 紧急情况 |
ERR | '3' | 错误情况 |
WARNING | '4' | 警告情况 |
NOTICE | '5' | 正常但重要的情况 |
INFO | '6' | 信息 |
DEBUG | '7' | 调试级别的消息 |
'S': synchronization message so that a reader can recover from a corrupt message by searching for the next sync message (not used currently).
sync_magic
: to be defined.'O': mark a dropout (lost logging messages) of a given duration in ms. Dropouts can occur e.g. if the device is not fast enough. 例如当设备不够快的情况下会出现丢包。
'I': information message. See above. 见上文。
'M': information message multi. See above. 见上文。
'P': parameter message. See above. 见上文。
'Q': parameter message. See above.
解析器的要求
一个有效的 ULog 解析器必须满足以下要求:
必须忽略未知消息 (但可以打印警告) 。
解析未来/未知的文件格式版本 (但可以打印警告) 。
必须拒绝解析包含未知不兼容位集 (
ulog_message_flag_bits_s
报文中的incompat_flags
) 的日志,这意味着日志包含解析器无法处理的突发改变。A parser must be able to correctly handle logs that end abruptly, in the middle of a message. The unfinished message should just be discarged. 未完成的报文应该丢弃。
对于附加数据:解析器可以假设数据部分存在,即在定义部分之后的位置有一个偏移点。
必须将附加数据视为常规数据部分的一部分。
已知的实现
PX4 Firmware: C++
hardfault_log module: append hardfault crash data.
pyulog:Python,使用 CLI 脚本的 Ulog 解析库。
FlightPlot: Java,日志绘图仪。
pyFlightAnalysis:Python,日志绘图仪和基于 pyulog 的三维可视化工具。
MAVLink:通过 MAVLink 进行 ULog 流的消息 (注意,不支持追加数据,至少不支持截断消息)。
QGroundControl:C++,通过 MAVLink 的 Ulog 流和最小的 GeoTagging。
mavlink-router:C++,通过 MAVLink 的 ULog 流。
MAVGAnalysis:Java,通过 MAVLink 的数据流和日志的绘制、分析。
PlotJuggler: 绘制日志和时间序列的 C++/Qt 应用。 自版本2.1.3支持 ULog。
ulogreader: Javascript, ULog reader and parser outputs log in JSON object format.
文件格式版本历史
版本 2 中的改变
Addition of ulog_message_info_multiple_header_s
and ulog_message_flag_bits_s
messages and the ability to append data to a log. This is used to add crash data to an existing log. If data is appended to a log that is cut in the middle of a message, it cannot be parsed with version 1 parsers. Other than that forward and backward compatibility is given if parsers ignore unknown messages. 这被用来给现有的日志添加损坏的数据。 如果从中间切开的报文数据被附加到日志中,这不能被版本 1 解析器解析。 除此之外,如果解析器忽略未知消息,则提供向前和向后的兼容性。
Last updated