How do you decode them time in a md5 header?

[blshep]

I am using verify=md5. Based on the structure in verify.h, the verify_header format is.....

The data written is shown below with the bytes that I think are the timestamp in seconds highlighted. Epoch and Unix timestamp converters don't work. What am I missing?

[sitsofe]

Hi @blshep,

(You didn't include the information that is requested in https://github.com/axboe/fio/blob/fio-3.38/REPORTING-BUGS#L12-L19 / https://github.com/axboe/fio/blob/fio-3.38/.github/SUPPORT.md so I will make assumptions that are convenient for me in answering your question)

The missing pieces of information are that fio will use a CPU clock as the clock source when possible (because that clock has the best granularity with the least overhead) and that clock does not start counting from the Unix epoch. Let's trace through fio's code:

In verify.c's __fill_hdr() we can see that the time time_sec and time_nsec fields of hdr are set from the io_u's start_time field:

	hdr->time_sec = io_u->start_time.tv_sec;
	hdr->time_nsec = io_u->start_time.tv_nsec;

In io_u.c's get_io_u() we can see the io_u->start_time being filled by a call to fio_gettime():

		if (!td->o.disable_lat)
			fio_gettime(&io_u->start_time, NULL);

In gettime.c we can find fio_gettime()'s definition:

#ifdef FIO_DEBUG_TIME
void fio_gettime(struct timespec *tp, void *caller)
#else
void fio_gettime(struct timespec *tp, void fio_unused *caller)
#endif
{
#ifdef FIO_DEBUG_TIME
	if (!caller)
		caller = __builtin_return_address(0);

	gtod_log_caller(caller);
#endif
	if (fio_unlikely(fio_gettime_offload(tp)))
		return;

	__fio_gettime(tp);
}

Finally we get to the definition of __fio_gettime():

static void __fio_gettime(struct timespec *tp)
{
	switch (fio_clock_source) {

which shows that fio can indeed use different clocks. On Linux x86_64 the different clocks available are:

• gettimeofday() Returns the number of seconds and microseconds since the Unix epoch.
• clock_gettime(CLOCK_MONOTONIC, ...) When the user requests a CLOCK_MONOTONIC clock on Linux it returns the seconds and nanoseconds the system has been running since it was booted.
• rdtsc register (which fio uses as its CPU clock) returns a cycle count since the processor was reset. fio has a calibration routine to work out how many cycles are in a millisecond. On top of that fio records the cycle count when it starts and subtracts that from the current cycles when the cpu clock is requested.

For comparison:

$ rm -f /tmp/fio.test; ./fio --filename=/tmp/fio.test --bs=64 --verify=md5 --size=64 --rw=write --zero_buffers --randseed=1 --clocksource=cpu --name=vhdrcpu && hexdump -C /tmp/fio.test 
[...]
00000000  ca ac 02 00 40 00 00 00  3a 6f cc 3c 1e 23 7e 1f  |....@...:o.<.#~.|
00000010  00 00 00 00 00 00 00 00  00 00 00 00 95 fe 4d 07  |..............M.|
00000020  01 00 00 00 cf c2 17 2e  75 62 de ba f7 d1 27 d4  |........ub....'.|
00000030  f8 35 bc 0c d4 85 93 cb  36 e9 f1 52 6a 6f db 8c  |.5......6..Rjo..|
00000040

Here we only have timestamp of 0x074dfe95 nanoseconds (approximately 123 milliseconds) and this roughly represents the time since fio started.

$ rm -f /tmp/fio.test; ./fio --filename=/tmp/fio.test --bs=64 --verify=md5 --size=64 --rw=write --zero_buffers --randseed=1 --clocksource=gettimeofday --name=vhdrgtod && hexdump -C /tmp/fio.test
[...]
% hexdump -C /tmp/fio.test 
00000000  ca ac 02 00 40 00 00 00  3a 6f cc 3c 1e 23 7e 1f  |....@...:o.<.#~.|
00000010  00 00 00 00 00 00 00 00  0e 22 9e 67 90 e0 8e 19  |.........".g....|
00000020  01 00 00 00 2f 34 b4 59  75 62 de ba f7 d1 27 d4  |..../4.Yub....'.|
00000030  f8 35 bc 0c d4 85 93 cb  36 e9 f1 52 6a 6f db 8c  |.5......6..Rjo..|
00000040

Here the timestamp is 0x679e220e + 0x198ee090/1000**3 seconds which is 1738416654.428794 . Let's see what date that converts to if we interpret it as a Unix time:

$ date --date @1738416654
Sat  1 Feb 13:30:54 GMT 2025

Which is when I ran the job.