Python 3 Support and Docker Update #136

jaycedowell · 2020-02-03T20:19:19Z

This PR adds support for Python 3 through a combination of changes to Bifrost itself as well as a new version of ctypesgen (which also addresses #115). It also updates the Docker GPU image to target Ubuntu 18/CUDA 10.2 (to address #129) and adds the tools in tools to the Python installation.

…o make sure that it works.

… a slightly newer version of Ubuntu?

… version. Will this be enough?

…bifrost into docker-gpu-update

coveralls · 2020-02-04T17:30:35Z

Coverage increased (+0.04%) to 61.701% when pulling 36b42a4 on docker-gpu-update into 5cdaa03 on master.

codecov-io · 2020-02-04T17:30:39Z

Codecov Report

Merging #136 into master will decrease coverage by 0.17%.
The diff coverage is 0%.

@@            Coverage Diff             @@
##           master     #136      +/-   ##
==========================================
- Coverage   58.91%   58.73%   -0.18%     
==========================================
  Files          63       64       +1     
  Lines        5140     5155      +15     
==========================================
  Hits         3028     3028              
- Misses       2112     2127      +15

Impacted Files	Coverage Δ
python/bifrost/romein.py	`0% <0%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fd754e6...602908f. Read the comment docs.

…ally is only a couple of lines.

…the Python installation.

… Updated src/Makefile to try to make the mapped path less fiddly.

codecov-io · 2020-03-18T20:12:56Z

Codecov Report

Merging #136 into master will increase coverage by 0.44%.
The diff coverage is 62.35%.

@@            Coverage Diff             @@
##           master     #136      +/-   ##
==========================================
+ Coverage   58.53%   58.97%   +0.44%     
==========================================
  Files          65       64       -1     
  Lines        5117     5155      +38     
==========================================
+ Hits         2995     3040      +45     
+ Misses       2122     2115       -7

Impacted Files	Coverage Δ
python/bifrost/__init__.py	`80.76% <25.00%> (+0.76%)`	⬆️
python/bifrost/portaudio.py	`4.02% <25.00%> (+0.64%)`	⬆️
python/bifrost/sigproc.py	`87.81% <25.00%> (+0.05%)`	⬆️
python/bifrost/sigproc2.py	`54.24% <33.33%> (+0.21%)`	⬆️
python/bifrost/blocks/sigproc.py	`24.88% <44.44%> (+1.05%)`	⬆️
python/bifrost/block.py	`89.86% <50.00%> (+0.06%)`	⬆️
python/bifrost/ring2.py	`86.70% <50.00%> (+0.04%)`	⬆️
python/bifrost/blocks/detect.py	`22.58% <60.00%> (+3.93%)`	⬆️
python/bifrost/pipeline.py	`84.47% <60.00%> (+0.09%)`	⬆️
python/bifrost/proclog.py	`31.42% <66.66%> (+4.15%)`	⬆️
... and 10 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b9140e8...c7a9782. Read the comment docs.

python/bifrost/addon/leda/blocks.py

telegraphic

FWIW I'm keen to see Py3 support, and don't see any showstoppers. Lots of changes but mainly minor Py3 compatibility. Can't comment on fir.cu changes as I haven't used it, and am assuming memory.cpp changes are solid, .

jaycedowell · 2020-05-21T23:03:21Z

Huh, the FDMT test seems to work now (and passes). Yay.

id -> id_ typos

…bifrost into docker-gpu-update

…can be run through a script without user intervention.

jaycedowell · 2020-06-04T03:50:21Z

@telegraphic I'm having problems running some of the "testbench" examples. test_guppi.py and test_guppi_reader.py don't seem to do anything. test_fft_detect.py initially would not run and still throws errors after I tweaked it.

print statements StringIO imports Exception raising

…_listen_data.py.

…t Jenkins failures.

…bifrost into docker-gpu-update

Docker gpu update

jaycedowell · 2020-07-07T20:41:39Z

Does anyone have any final comments on this PR or opinions about updating the Bifrost version to 0.9.0? If so, comment now or forever hold your peace.

jaycedowell added 18 commits August 9, 2019 13:09

Update the .travis.yml page to include a call to 'make docker' just t…

a2bca23

…o make sure that it works.

Updated Dockerfile.gpu to use a safe base image. Maybe we want to use…

3b2b365

… a slightly newer version of Ubuntu?

Moved up to Ubuntu 16.04 for the GPU Dockerfile.

e00c474

Trying out a new version of ctypesgen and Python3 support.

c7e6e94

Looks like things were moved around quite a bit in this new ctypesgen…

cb759db

… version. Will this be enough?

Updated the version of ctypesgen used in the Dockerfile.cpu.

a57d88b

Attempting to test on an additional version of PyPy (3.5-7.0).

fe14238

Trying a different PyPy3.5.

1e7d370

Trying yet another PyPy version for Python3 support.

2593cb7

Moved more Bifrost modules over to print_function.

08d7c53

Updated the verion of pypy used for the Python2.7 testing.

d25155d

Trying Ubuntu 18.04 just to see what happens.

c3fdbc9

Updated the Python3 version to 3.6.

68d5de7

Cleaned up range/xrange in a Python2 and Python3 compatible way.

de20104

And missed bifrost.blocks.sigproc.

fd754e6

Merge branch 'master' into docker-gpu-update

41620f5

Moved the Docker GPU images over to CUDA 10.2.

e8a9f4a

Merge branch 'docker-gpu-update' of https://github.com/ledatelescope/…

8125bec

…bifrost into docker-gpu-update

jaycedowell added 5 commits February 4, 2020 11:35

Moved the contents of python/wrap.py into python/Makefile since it re…

5af2b74

…ally is only a couple of lines.

Did that do anything?

64d50e4

Updated the setup.py script to include the tools in tools as part of …

a447d8b

…the Python installation.

Cleaned up a few compiler warnings in memory.cpp under CUDA 10+.

06f75ab

Actually fixed the CUDA 10.2 warnings in memory.cpp.

602908f

jaycedowell added a commit to jaycedowell/bifrost that referenced this pull request Mar 18, 2020

Clean up some compiler warnings in memory.cpp a la PR lwa-project#136.…

497057a

… Updated src/Makefile to try to make the mapped path less fiddly.

Cleaned up some compiler warnings in fir.cu.

c0d9601

telegraphic reviewed Apr 15, 2020

View reviewed changes

python/bifrost/addon/leda/blocks.py Outdated Show resolved Hide resolved

telegraphic reviewed Apr 15, 2020

View reviewed changes

Added in test_fdmt to the Jenkins tests.

2eaa2b7

jack-h and others added 5 commits June 2, 2020 03:35

id -> id_ typos

2c2dbab

Merge pull request #141 from realtimeradio/id_fix

1408f54

id -> id_ typos

More prints that needed to be converted.

0b2603f

Merge branch 'docker-gpu-update' of https://github.com/ledatelescope/…

a949c43

…bifrost into docker-gpu-update

Added a -y option to download_breakthrough_listen_data.py so that it …

57a2d9f

…can be run through a script without user intervention.

jack-h and others added 17 commits June 4, 2020 10:27

More Python3 imports

caa2359

Py2 -> 3 changes

f324f71

print statements StringIO imports Exception raising

I have no idea why test_fft_detect.py always fails.

73ffc64

Pulled in the recent changes to master.

53fa44d

Merge branch 'docker-gpu-update' into docker-gpu-update

39d3acc

Python2/3 compatibility catch

c94947b

Fixed a few path problems in test_guppi*.py and download_breakthrough…

79db26c

…_listen_data.py.

I still don't understand test_fft_detect.py.

03c3a68

Cleaned up after test_fft*.py and added in the jenkins.sh test launcher.

f0f3a3c

Typo in jenkins.sh. Plus, a test to see if I can track down the recen…

4ad3286

…t Jenkins failures.

Merge branch 'docker-gpu-update' of https://github.com/ledatelescope/…

9d5e40f

…bifrost into docker-gpu-update

Move sys import to before its first use

6eeb92f

Worked on fixing problems uncovered in the testbench tests.

3ed8c02

A more robust fix for DataType.py.

c2b96df

Ugh.

746ffd1

Let's see if we can get the coverage in the testbench test counted.

e0fd0f8

Merge pull request #142 from realtimeradio/docker-gpu-update

fe96233

Docker gpu update

jaycedowell added 2 commits July 21, 2020 14:58

Version bump.

9de5403

Follow redirects to get to bf_test_files.tar.gz.

36b42a4

jaycedowell merged commit a53b35b into master Jul 23, 2020

jaycedowell deleted the docker-gpu-update branch July 23, 2020 15:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Python 3 Support and Docker Update #136

Python 3 Support and Docker Update #136

jaycedowell commented Feb 3, 2020 •

edited

Loading

coveralls commented Feb 4, 2020 •

edited

Loading

codecov-io commented Feb 4, 2020 •

edited

Loading

codecov-io commented Mar 18, 2020 •

edited

Loading

telegraphic left a comment

jaycedowell commented May 21, 2020

jaycedowell commented Jun 4, 2020

jaycedowell commented Jul 7, 2020

Python 3 Support and Docker Update #136

Python 3 Support and Docker Update #136

Conversation

jaycedowell commented Feb 3, 2020 • edited Loading

coveralls commented Feb 4, 2020 • edited Loading

codecov-io commented Feb 4, 2020 • edited Loading

Codecov Report

codecov-io commented Mar 18, 2020 • edited Loading

Codecov Report

telegraphic left a comment

Choose a reason for hiding this comment

jaycedowell commented May 21, 2020

jaycedowell commented Jun 4, 2020

jaycedowell commented Jul 7, 2020

jaycedowell commented Feb 3, 2020 •

edited

Loading

coveralls commented Feb 4, 2020 •

edited

Loading

codecov-io commented Feb 4, 2020 •

edited

Loading

codecov-io commented Mar 18, 2020 •

edited

Loading