affsign -k: segmentation fault in s390x, ppc64 and sparc64 (Debian) #41

eribertomota · 2020-07-01T01:52:14Z

In my last Debian revision for afflib, I re-enabled the upstream tests (disabled since 2010). The package was built in several architectures but failed to build from source in three archs: s390x, ppc64 and sparc64. I attached the build logs but you can see a relevant part below:

Signing AFF file...
affsign -k /tmp/basevP50b.agent.pem /tmp/basevP50b.evidence.aff
Segmentation fault
affsign failed
FAIL test_signing.sh (exit status: 1)

============================================================================
Testsuite summary for AFFLIB 3.7.18
============================================================================
# TOTAL: 5
# PASS:  4
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0
============================================================================

I have SSH access to all machines and I can help to do tests. In a s390x, running the command by hand, I can see:

./affsign -k /tmp/basevP50b.agent.pem /tmp/basevP50b.evidence.aff
Signing segments...
Calculating BOM for segment badflag...   
Calculating BOM for segment badsectors...   
Calculating BOM for segment afflib_version...   
Calculating BOM for segment aff_file_type...   
Calculating BOM for segment acquisition_commandline...   
Calculating BOM for segment pagesize...   
Calculating BOM for segment sectorsize...   
Calculating BOM for segment page0...   
Calculating BOM for segment page1...   
Segmentation fault

Please, let me know if you need more details or tests.

Taking advantage of this message, I would like to ask for use other source to generate the rawevidence.raw file. Currently, it uses the content of /usr/share/dict/. It is bad for me because I need force the build system to install dictionaries. See below:

TEST ./test_signing.sh
=== MAKING THE TEST FILES ===
Making the random ISO rawevidence.raw
cat: '/usr/share/dict/*': No such file or directory
cat: '/usr/share/dict/*': No such file or directory
cat: '/usr/share/dict/*': No such file or directory
cat: '/usr/share/dict/*': No such file or directory
cat: '/usr/share/dict/*': No such file or directory
cat: '/usr/share/dict/*': No such file or directory
cat: '/usr/share/dict/*': No such file or directory
cat: '/usr/share/dict/*': No such file or directory
cat: '/usr/share/dict/*': No such file or directory
cat: '/usr/share/dict/*': No such file or directory
-rw-r--r-- 1 buildd buildd 35127296 Jun 30 03:31 rawevidence.raw
MD5(rawevidence.raw)= a02ae7822b19f78ee40f18d3740567f4

Thanks a lot in advance.

Regards,

Eriberto

afflib-logs.tar.gz

The text was updated successfully, but these errors were encountered:

sshock · 2020-07-01T03:43:40Z

Hi @eribertomota . My suspicion is an endian-ness bug somewhere, but that is just a guess. Can you get me a call stack where it seg faults?

For the rawevidence.raw, I am open to suggestions on how to change that script. I think we could just have it grab files from somewhere else like /bin/ instead. Would that work?

eribertomota · 2020-07-01T03:56:25Z

Hi @eribertomota . My suspicion is an endian-ness bug somewhere, but that is just a guess. Can you get me a call stack where it seg faults?

Thanks for your very quick reply. Can you guide me how to get a call stack?

For the rawevidence.raw, I am open to suggestions on how to change that script. I think we could just have it grab files from somewhere else like /bin/ instead. Would that work?

I think the rawevidence.raw must be ASCII only (because the old upstream focused in dictionaries). I suggest to use local .cpp files as source or a file from /etc. What you think about it?

sshock · 2020-07-01T04:06:00Z

Thanks for your very quick reply. Can you guide me how to get a call stack?

If you run aftest inside gdb (e.g., gdb lib/aftest and then inside gdb type run and hit enter), then inside gdb after it seg faults just type bt and hit enter and that should spit out a back trace.

I think the rawevidence.raw must be ASCII only (because the old upstream focused in dictionaries). I suggest to use local .cpp files as source or a file from /etc. What you think about it?

I can't see anything that indicates they have to be text files, but I like your idea better, so I'll try that.

eribertomota · 2020-07-01T04:10:31Z

aftest or affsign?

sshock · 2020-07-01T04:20:50Z

oh yeah, sorry; affsign

sshock · 2020-07-01T04:23:17Z

This worked for me (I had to use .libs/affsign because affsign ends up being a libtool wrapper script or somethign):

gdb .libs/affsign
set args -k /tmp/basekIHqK.agent.pem /tmp/basekIHqK.evidence.aff
run

eribertomota · 2020-07-01T04:26:22Z

In s390x:

/home/eriberto/afflib/afflib-3.7.18/tools/.libs/affsign: error while loading shared libraries: libafflib.so.0: cannot open shared object file: No such file or directory
[Inferior 1 (process 13801) exited with code 0177]

eribertomota · 2020-07-01T04:28:45Z

Maybe I need install afflib in environment...

sshock · 2020-07-01T04:30:45Z

fyi, I just put in the fix to make it not rely on /usr/share/dict: 15ebb83

eribertomota · 2020-07-01T04:31:32Z

Ok, I did a test in my local machine. I need install afflib and use a prebuilt tmp/name.pem. The problem is I don't have permission to install afflib in remote machine... Is there a way to use a local libafflib.so.0?

eribertomota · 2020-07-01T04:32:03Z

fyi, I just put in the fix to make it not rely on /usr/share/dict: 15ebb83

Good!

eribertomota · 2020-07-01T04:36:27Z

I just installed afflib in remote. A moment please...

eribertomota · 2020-07-01T04:41:10Z

Well... I don't understand...

$ gdb .libs/affsign
(gdb) set args -k /tmp/baseD7ezF.agent.pem /tmp/baseD7ezF.evidence.aff
(gdb) run
Starting program: /home/eriberto/afflib/afflib-3.7.18/tools/.libs/affsign -k /tmp/baseD7ezF.agent.pem /tmp/baseD7ezF.evidence.aff
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/s390x-linux-gnu/libthread_db.so.1".
Signing segments...
Calculating BOM for segment badflag...   
Calculating BOM for segment badsectors...   
Calculating BOM for segment afflib_version...   
Calculating BOM for segment aff_file_type...   
Calculating BOM for segment acquisition_commandline...   
Calculating BOM for segment pagesize...   
Calculating BOM for segment sectorsize...   
Calculating BOM for segment page0...   
Calculating BOM for segment page1...   
Calculating BOM for segment page2...   
Calculating BOM for segment imagesize...   
Calculating BOM for segment md5...   
Calculating BOM for segment sha1...   
Calculating BOM for segment image_gid...   
Calculating BOM for segment acquisition_date...   
Calculating BOM for segment cert-sha256...   
Calculating BOM for segment badflag/sha256...   
Calculating BOM for segment badsectors/sha256...   
Calculating BOM for segment afflib_version/sha256...   
Calculating BOM for segment aff_file_type/sha256...   
Calculating BOM for segment acquisition_commandline/sha256...   
Calculating BOM for segment pagesize/sha256...   
Calculating BOM for segment sectorsize/sha256...   
Calculating BOM for segment page0/sha256...   
Calculating BOM for segment page1/sha256...   
Calculating BOM for segment page2/sha256...   
Calculating BOM for segment imagesize/sha256...   
Calculating BOM for segment md5/sha256...   
Calculating BOM for segment sha1/sha256...   
Calculating BOM for segment image_gid/sha256...   
Calculating BOM for segment acquisition_date/sha256...   
Calculating BOM for segment cert-sha256/sha256...   
                                                    
[Inferior 1 (process 55096) exited normally]
(gdb) bt
No stack.
(gdb)

No fail, no stack.

eribertomota · 2020-07-01T04:42:44Z

However, I ran the build command again and I got the same error.

sshock · 2020-07-01T04:50:45Z

Hmm, that is very odd for it to work outside of gdb but not inside, crazy!

How about this. Can you make it dump a core file? I think you can enable that by running ulimit -c unlimited. Then go ahead and run ./affsign -k /tmp/baseD7ezF.agent.pem /tmp/baseD7ezF.evidence.aff. It should create a core file. Then you can open the core file with gdb using gdb .libs/affsign core. Then I think you will be able to immediately run bt in gdb to get the call stack.

eribertomota · 2020-07-01T04:54:22Z

I created a valgrind for you... I will try gdb.

valgrind.gz

eribertomota · 2020-07-01T04:57:04Z

(sid_s390x-dchroot)eriberto@zelenka:~/afflib/afflib-3.7.18/tools$ gdb .libs/affsign core
GNU gdb (Debian 9.2-1) 9.2
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "s390x-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from .libs/affsign...
[New LWP 1877]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/s390x-linux-gnu/libthread_db.so.1".
Core was generated by `/home/eriberto/afflib/afflib-3.7.18/tools/.libs/affsign -k /tmp/baseeqOfU.agent'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x000003ff8581bd58 in af_get_page (af=0x2aa2cfa7040, pagenum=<optimized out>, data=<optimized out>, 
    bytes=0x3ffe4dfe3b0) at afflib_pages.cpp:308
308     afflib_pages.cpp: No such file or directory.
(gdb) bt
#0  0x000003ff8581bd58 in af_get_page (af=0x2aa2cfa7040, pagenum=<optimized out>, data=<optimized out>, 
    bytes=0x3ffe4dfe3b0) at afflib_pages.cpp:308
#1  0x000002aa2b482fa4 in affsign (fn=<optimized out>) at affsign.cpp:159
#2  0x000003ff84fab5b6 in __libc_start_main (main=0x2aa2b4828d8 <main(int, char**)>, argc=<optimized out>, 
    argv=0x3ffe4dff288, init=<optimized out>, fini=0x2aa2b484730 <__libc_csu_fini>, 
    rtld_fini=0x3ff858904e8 <_dl_fini>, stack_end=0x3ffe4dff1d0) at libc-start.c:308
#3  0x000002aa2b482b34 in _start () at affsign.cpp:305
(gdb)

eribertomota · 2020-07-01T04:58:17Z

---> 308 afflib_pages.cpp: No such file or directory ???

sshock · 2020-07-01T05:02:47Z

I created a valgrind for you... I will try gdb.

valgrind.gz

wow, so many errors!

sshock · 2020-07-01T05:06:22Z

---> 308 afflib_pages.cpp: No such file or directory ???

I think the seg fault happened on line 308 of afflib_pages.cpp, but the No such file or directory just means it couldn't open the source code for some reason.

Here's lines 306-309:

306:    size_t bytes_left_in_sector = (SECTOR_SIZE - (*bytes % SECTOR_SIZE)) % SECTOR_SIZE;
307:    for(size_t i=0;i<bytes_left_in_sector;i++){
308:        data[*bytes + i] = 0;
309:    }

sshock · 2020-07-01T05:34:46Z

I'm scouring the code here but not seeing the problem yet. Seems likely to be something such as the data buffer not getting allocated to the full size it should be (pagesize).

sshock · 2020-07-01T06:04:20Z

can you tell me what sizeof(long) is on those systems?

eribertomota · 2020-07-01T06:10:03Z

How to get it?

eribertomota · 2020-07-01T06:12:31Z

s390x
$ cpp -dD /dev/null | grep SIZEOF_LONG
#define SIZEOF_LONG 8

hurd
$ cpp -dD /dev/null | grep SIZEOF_LONG
#define SIZEOF_LONG 4

sshock · 2020-07-01T06:13:21Z

ok, yeah I think I found the problem; one minute...

eribertomota · 2020-07-01T06:14:13Z

good

sshock · 2020-07-01T06:17:17Z

please try changing line 244 in lib/afflib_pages.cpp, from this:

*bytes = ntohl(*(long *)compressed_data);

to this:

*bytes = ntohl(*(uint32_t *)compressed_data);

and let me know if it runs better

eribertomota · 2020-07-01T06:25:32Z

Worked fine!

sshock · 2020-07-01T06:28:07Z

awesome! committing the fix and adding in a sanity test to go with it now...

eribertomota · 2020-07-01T06:28:44Z

If you want I can prepare a patch and send to Debian experimental to test in all archs before you release a new version.

sshock · 2020-07-01T06:29:23Z

@eribertomota yes, that would be great, thanks!

here is the commit with the fix, which is also in master now: e386977

eribertomota · 2020-07-01T06:32:50Z

I will send master to Debian. For Debian it is 3.7.18+git20200701.e386977

sshock · 2020-07-01T06:37:46Z

remind me when this is all done to create a new version (tag), since it has been a while

eribertomota · 2020-07-01T07:11:13Z

Sure!

eribertomota mentioned this issue Jul 1, 2020

aftest: bugs.aff: Invalid argument in HURD (Debian) #42

Closed

sshock closed this as completed in e386977 Jul 1, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

affsign -k: segmentation fault in s390x, ppc64 and sparc64 (Debian) #41

affsign -k: segmentation fault in s390x, ppc64 and sparc64 (Debian) #41

eribertomota commented Jul 1, 2020

sshock commented Jul 1, 2020

eribertomota commented Jul 1, 2020

sshock commented Jul 1, 2020

eribertomota commented Jul 1, 2020

sshock commented Jul 1, 2020

sshock commented Jul 1, 2020

eribertomota commented Jul 1, 2020

eribertomota commented Jul 1, 2020

sshock commented Jul 1, 2020

eribertomota commented Jul 1, 2020

eribertomota commented Jul 1, 2020

eribertomota commented Jul 1, 2020

eribertomota commented Jul 1, 2020

eribertomota commented Jul 1, 2020

sshock commented Jul 1, 2020

eribertomota commented Jul 1, 2020

eribertomota commented Jul 1, 2020

eribertomota commented Jul 1, 2020

sshock commented Jul 1, 2020

sshock commented Jul 1, 2020

sshock commented Jul 1, 2020

sshock commented Jul 1, 2020

eribertomota commented Jul 1, 2020

eribertomota commented Jul 1, 2020

sshock commented Jul 1, 2020

eribertomota commented Jul 1, 2020

sshock commented Jul 1, 2020

eribertomota commented Jul 1, 2020

sshock commented Jul 1, 2020

eribertomota commented Jul 1, 2020

sshock commented Jul 1, 2020

eribertomota commented Jul 1, 2020

sshock commented Jul 1, 2020

eribertomota commented Jul 1, 2020

affsign -k: segmentation fault in s390x, ppc64 and sparc64 (Debian) #41

affsign -k: segmentation fault in s390x, ppc64 and sparc64 (Debian) #41

Comments

eribertomota commented Jul 1, 2020

sshock commented Jul 1, 2020

eribertomota commented Jul 1, 2020

sshock commented Jul 1, 2020

eribertomota commented Jul 1, 2020

sshock commented Jul 1, 2020

sshock commented Jul 1, 2020

eribertomota commented Jul 1, 2020

eribertomota commented Jul 1, 2020

sshock commented Jul 1, 2020

eribertomota commented Jul 1, 2020

eribertomota commented Jul 1, 2020

eribertomota commented Jul 1, 2020

eribertomota commented Jul 1, 2020

eribertomota commented Jul 1, 2020

sshock commented Jul 1, 2020

eribertomota commented Jul 1, 2020

eribertomota commented Jul 1, 2020

eribertomota commented Jul 1, 2020

sshock commented Jul 1, 2020

sshock commented Jul 1, 2020

sshock commented Jul 1, 2020

sshock commented Jul 1, 2020

eribertomota commented Jul 1, 2020

eribertomota commented Jul 1, 2020

sshock commented Jul 1, 2020

eribertomota commented Jul 1, 2020

sshock commented Jul 1, 2020

eribertomota commented Jul 1, 2020

sshock commented Jul 1, 2020

eribertomota commented Jul 1, 2020

sshock commented Jul 1, 2020

eribertomota commented Jul 1, 2020

sshock commented Jul 1, 2020

eribertomota commented Jul 1, 2020