Skip to content

Commit c77adce

Browse files
hmstepaneklrafeeiaaeabdomergify[bot]TimPansino
authored
Merge main (#874)
* Exclude command line functionality from test coverage (#855) * FIX: resilient environment settings (#825) if the application uses generalimport to manage optional depedencies, it's possible that generalimport.MissingOptionalDependency is raised. In this case, we should not report the module as it is not actually loaded and is not a runtime dependency of the application. Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Hannah Stepanek <[email protected]> * Replace drop_transaction logic by using transaction context manager (#832) * Replace drop_transaction call * [Mega-Linter] Apply linters fixes * Empty commit to start tests * Change logic in BG Wrappers --------- Co-authored-by: lrafeei <[email protected]> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Upgrade to Pypy38 for TypedDict (#861) * Fix base branch * Revert tox dependencies * Replace all pypy37 with pypy38 * Remove action.yml file * Push Empty Commit * Fix skip_missing_interpreters behavior * Fix skip_missing_interpreters behavior * Pin dev CI image sha * Remove unsupported Tornado tests * Add latest tests to Tornado * Remove pypy38 (for now) --------- Co-authored-by: Tim Pansino <[email protected]> * Add profile_trace testing (#858) * Include isort stdlibs for determining stdlib modules * Use isort & sys to eliminate std & builtin modules Previously, the logic would fail to identify third party modules installed within the local user socpe. This fixes that issue by skipping builtin and stdlib modules by name, instead of attempting to identify third party modules based on file paths. * Handle importlib_metadata.version being a callable * Add isort into third party notices * [Mega-Linter] Apply linters fixes * Remove Python 2.7 and pypy2 testing (#835) * Change setup-python to @v2 for py2.7 * Remove py27 and pypy testing * Fix syntax errors * Fix comma related syntax errors * Fix more issues in tox * Remove gearman test * Containerized CI Pipeline (#836) * Revert "Remove Python 2.7 and pypy2 testing (#835)" This reverts commit abb6405. * Containerize CI process * Publish new docker container for CI images * Rename github actions job * Copyright tag scripts * Drop debug line * Swap to new CI image * Move pip install to just main python * Remove libcurl special case from tox * Install special case packages into main image * Remove unused packages * Remove all other triggers besides manual * Add make run command * Cleanup small bugs * Fix CI Image Tagging (#838) * Correct templated CI image name * Pin pypy2.7 in image * Fix up scripting * Temporarily Restore Old CI Pipeline (#841) * Restore old pipelines * Remove python 2 from setup-python * Rework CI Pipeline (#839) Change pypy to pypy27 in tox. Fix checkout logic Pin tox requires * Fix Tests on New CI (#843) * Remove non-root user * Test new CI image * Change pypy to pypy27 in tox. * Fix checkout logic * Fetch git tags properly * Pin tox requires * Adjust default db settings for github actions * Rename elasticsearch services * Reset to new pipelines * [Mega-Linter] Apply linters fixes * Fix timezone * Fix docker networking * Pin dev image to new sha * Standardize gearman DB settings * Fix elasticsearch settings bug * Fix gearman bug * Add missing odbc headers * Add more debug messages * Swap out dev ci image * Fix required virtualenv version * Swap out dev ci image * Swap out dev ci image * Remove aioredis v1 for EOL * Add coverage paths for docker container * Unpin ci container --------- Co-authored-by: TimPansino <[email protected]> * Trigger tests * Add testing for profile trace. * [Mega-Linter] Apply linters fixes * Ignore __call__ from coverage on profile_trace. * [Mega-Linter] Apply linters fixes --------- Co-authored-by: Hannah Stepanek <[email protected]> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: hmstepanek <[email protected]> Co-authored-by: Lalleh Rafeei <[email protected]> Co-authored-by: Timothy Pansino <[email protected]> Co-authored-by: TimPansino <[email protected]> Co-authored-by: umaannamalai <[email protected]> * Add Transaction API Tests (#857) * Test for suppress_apdex_metric * Add custom_metrics tests * Add distributed_trace_headers testing in existing tests * [Mega-Linter] Apply linters fixes * Remove redundant if-statement * Ignore deprecated transaction function from coverage * [Mega-Linter] Apply linters fixes * Push empty commit * Update newrelic/api/transaction.py --------- Co-authored-by: lrafeei <[email protected]> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Timothy Pansino <[email protected]> Co-authored-by: Uma Annamalai <[email protected]> * Add tests for jinja2. (#842) * Add tests for jinja2. * [Mega-Linter] Apply linters fixes * Update tox.ini Co-authored-by: Timothy Pansino <[email protected]> --------- Co-authored-by: umaannamalai <[email protected]> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Timothy Pansino <[email protected]> * Add tests for newrelic/config.py (#860) Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Fix starlette testing matrix for updated behavior. (#869) Co-authored-by: Lalleh Rafeei <[email protected]> Co-authored-by: Hannah Stepanek <[email protected]> Co-authored-by: Uma Annamalai <[email protected]> * Correct Serverless Distributed Tracing Logic (#870) * Fix serverless logic for distributed tracing * Test stubs * Collapse testing changes * Add negative testing to regular DT test suite * Apply linter fixes * [Mega-Linter] Apply linters fixes --------- Co-authored-by: TimPansino <[email protected]> * Fix Kafka CI (#863) * Reenable kafka testing * Add kafka dev lib * Sync install python with devcontainer * Fix kafka local host setting * Drop set -u flag * Pin CI image dev sha * Add parallel flag to kafka * Fix proper exit status * Build librdkafka from source * Updated dev image sha * Remove coverage exclusions * Add new options to better emulate GHA * Reconfigure kafka networking Co-authored-by: Hannah Stepanek <[email protected]> * Fix kafka ports on GHA * Run kafka tests serially * Separate kafka consumer groups * Put CI container makefile back * Remove confluent kafka Py27 for latest * Roll back ubuntu version update * Update dev ci sha --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Hannah Stepanek <[email protected]> * Change image tag to latest (#871) * Change image tag to latest * Use built sha * Fixup * Replace w/ latest * Add full version for pypy3.8 to tox (#872) * Add full version for pypy3.8 * Remove solrpy from tests * Fix merge conflict * Fix tests for scikit-learn >= 1.3.0 In 1.3.0 sklearn renamed fit to _fit in BaseDecisionTree. * Add gfortran to container * Use ci image sha * Add pkg-config * New CI build --------- Co-authored-by: Lalleh Rafeei <[email protected]> Co-authored-by: Ahmed Helil <[email protected]> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: lrafeei <[email protected]> Co-authored-by: Tim Pansino <[email protected]> Co-authored-by: Uma Annamalai <[email protected]> Co-authored-by: hmstepanek <[email protected]> Co-authored-by: Timothy Pansino <[email protected]> Co-authored-by: TimPansino <[email protected]> Co-authored-by: umaannamalai <[email protected]>
1 parent db915d0 commit c77adce

27 files changed

+1245
-548
lines changed

.github/actions/setup-python-matrix/action.yml

-50
This file was deleted.

.github/containers/Dockerfile

+18-4
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
2626
freetds-common \
2727
freetds-dev \
2828
gcc \
29+
gfortran \
2930
git \
3031
libbz2-dev \
3132
libcurl4-openssl-dev \
@@ -43,6 +44,7 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
4344
make \
4445
odbc-postgresql \
4546
openssl \
47+
pkg-config \
4648
python2-dev \
4749
python3-dev \
4850
python3-pip \
@@ -55,10 +57,22 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
5557
zlib1g-dev && \
5658
rm -rf /var/lib/apt/lists/*
5759

60+
# Build librdkafka from source
61+
ARG LIBRDKAFKA_VERSION=2.1.1
62+
RUN cd /tmp && \
63+
wget https://github.com/confluentinc/librdkafka/archive/refs/tags/v${LIBRDKAFKA_VERSION}.zip -O ./librdkafka.zip && \
64+
unzip ./librdkafka.zip && \
65+
rm ./librdkafka.zip && \
66+
cd ./librdkafka-${LIBRDKAFKA_VERSION} && \
67+
./configure && \
68+
make all install && \
69+
cd /tmp && \
70+
rm -rf ./librdkafka-${LIBRDKAFKA_VERSION}
71+
5872
# Setup ODBC config
59-
RUN sed -i 's/Driver=psqlodbca.so/Driver=\/usr\/lib\/x86_64-linux-gnu\/odbc\/psqlodbca.so/g' /etc/odbcinst.ini && \
60-
sed -i 's/Driver=psqlodbcw.so/Driver=\/usr\/lib\/x86_64-linux-gnu\/odbc\/psqlodbcw.so/g' /etc/odbcinst.ini && \
61-
sed -i 's/Setup=libodbcpsqlS.so/Setup=\/usr\/lib\/x86_64-linux-gnu\/odbc\/libodbcpsqlS.so/g' /etc/odbcinst.ini
73+
RUN sed -i 's|Driver=psqlodbca.so|Driver=/usr/lib/x86_64-linux-gnu/odbc/psqlodbca.so|g' /etc/odbcinst.ini && \
74+
sed -i 's|Driver=psqlodbcw.so|Driver=/usr/lib/x86_64-linux-gnu/odbc/psqlodbcw.so|g' /etc/odbcinst.ini && \
75+
sed -i 's|Setup=libodbcpsqlS.so|Setup=/usr/lib/x86_64-linux-gnu/odbc/libodbcpsqlS.so|g' /etc/odbcinst.ini
6276

6377
# Set the locale
6478
RUN locale-gen --no-purge en_US.UTF-8
@@ -79,7 +93,7 @@ RUN echo 'eval "$(pyenv init -)"' >>$HOME/.bashrc && \
7993
pyenv update
8094

8195
# Install Python
82-
ARG PYTHON_VERSIONS="3.10 3.9 3.8 3.7 3.11 2.7 pypy2.7-7.3.11 pypy3.7"
96+
ARG PYTHON_VERSIONS="3.10 3.9 3.8 3.7 3.11 2.7 pypy2.7-7.3.12 pypy3.8-7.3.11"
8397
COPY --chown=1000:1000 --chmod=+x ./install-python.sh /tmp/install-python.sh
8498
COPY ./requirements.txt /requirements.txt
8599
RUN /tmp/install-python.sh && \

.github/containers/Makefile

+5-1
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,9 @@ default: test
2222
.PHONY: build
2323
build:
2424
@# Perform a shortened build for testing
25-
@docker build --build-arg='PYTHON_VERSIONS=3.10 2.7' $(MAKEFILE_DIR) -t ghcr.io/newrelic/newrelic-python-agent-ci:local
25+
@docker build $(MAKEFILE_DIR) \
26+
-t ghcr.io/newrelic/newrelic-python-agent-ci:local \
27+
--build-arg='PYTHON_VERSIONS=3.10 2.7'
2628

2729
.PHONY: test
2830
test: build
@@ -38,7 +40,9 @@ run: build
3840
@docker run --rm -it \
3941
--mount type=bind,source="$(REPO_ROOT)",target=/home/github/python-agent \
4042
--workdir=/home/github/python-agent \
43+
--add-host=host.docker.internal:host-gateway \
4144
-e NEW_RELIC_HOST="${NEW_RELIC_HOST}" \
4245
-e NEW_RELIC_LICENSE_KEY="${NEW_RELIC_LICENSE_KEY}" \
4346
-e NEW_RELIC_DEVELOPER_MODE="${NEW_RELIC_DEVELOPER_MODE}" \
47+
-e GITHUB_ACTIONS="true" \
4448
ghcr.io/newrelic/newrelic-python-agent-ci:local /bin/bash

.github/containers/install-python.sh

+2-5
Original file line numberDiff line numberDiff line change
@@ -13,10 +13,7 @@
1313
# See the License for the specific language governing permissions and
1414
# limitations under the License.
1515

16-
set -e
17-
18-
SCRIPT_DIR=$(dirname "$0")
19-
PIP_REQUIREMENTS=$(cat /requirements.txt)
16+
set -eo pipefail
2017

2118
main() {
2219
# Coerce space separated string to array
@@ -50,7 +47,7 @@ main() {
5047
pyenv global ${PYENV_VERSIONS[@]}
5148

5249
# Install dependencies for main python installation
53-
pyenv exec pip install --upgrade $PIP_REQUIREMENTS
50+
pyenv exec pip install --upgrade -r /requirements.txt
5451
}
5552

5653
main

0 commit comments

Comments
 (0)