Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: aadarshadhakalg/restc-cpp
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: master
Choose a base ref
...
head repository: jgaa/restc-cpp
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: master
Choose a head ref
Can’t automatically merge. Don’t worry, you can still create the pull request.
Loading
Showing with 1,898 additions and 1,273 deletions.
  1. +107 −0 .github/workflows/ci.yaml
  2. +80 −35 CMakeLists.txt
  3. +35 −12 README.md
  4. +4 −3 ci/jenkins/{Dockerfile.debian-stretch → Dockefile.debian-bookworm}
  5. +1 −1 ci/jenkins/Dockefile.debian-buster
  6. +0 −26 ci/jenkins/Dockefile.ubuntu-bionic
  7. +2 −1 ci/jenkins/Dockefile.ubuntu-jammy
  8. +2 −3 ci/jenkins/Dockefile.ubuntu-xenial
  9. +17 −9 ci/jenkins/Dockerfile.fedora
  10. +24 −0 ci/jenkins/Dockerfile.ubuntu-bionic
  11. +23 −0 ci/jenkins/Dockerfile.ubuntu-noble
  12. +199 −244 ci/jenkins/Jenkinsfile.groovy
  13. +22 −0 ci/vcpkg/vcpkg.json
  14. +28 −29 cmake_scripts/external-projects.cmake
  15. +26 −10 create-and-run-containers.sh
  16. +11 −14 examples/logip/logip.cpp
  17. +1 −1 include/restc-cpp/DataReader.h
  18. +2 −2 include/restc-cpp/DataReaderStream.h
  19. +2 −2 include/restc-cpp/DataWriter.h
  20. +10 −9 include/restc-cpp/IteratorFromJsonSerializer.h
  21. +2 −1 include/restc-cpp/RapidJsonReader.h
  22. +5 −5 include/restc-cpp/RequestBuilder.h
  23. +1 −0 include/restc-cpp/SerializeJson-cpp17.h
  24. +1 −0 include/restc-cpp/SerializeJson.h
  25. +5 −4 include/restc-cpp/Socket.h
  26. +217 −0 include/restc-cpp/boost_compatibility.h
  27. +9 −1 include/restc-cpp/helper.h
  28. +14 −2 include/restc-cpp/logging.h
  29. +26 −24 include/restc-cpp/restc-cpp.h
  30. +18 −1 include/restc-cpp/test_helper.h
  31. +9 −0 include/restc-cpp/url_encode.h
  32. +21 −22 src/ChunkedReaderImpl.cpp
  33. +5 −7 src/ChunkedWriterImpl.cpp
  34. +80 −28 src/ConnectionPoolImpl.cpp
  35. +12 −11 src/DataReaderStream.cpp
  36. +4 −3 src/IoReaderImpl.cpp
  37. +3 −5 src/IoWriterImpl.cpp
  38. +2 −4 src/NoBodyReaderImpl.cpp
  39. +6 −7 src/PlainReaderImpl.cpp
  40. +4 −4 src/PlainWriterImpl.cpp
  41. +25 −16 src/ReplyImpl.cpp
  42. +5 −3 src/ReplyImpl.h
  43. +6 −10 src/RequestBodyFileImpl.cpp
  44. +7 −14 src/RequestBodyStringImpl.cpp
  45. +109 −100 src/RequestImpl.cpp
  46. +34 −29 src/RestClientImpl.cpp
  47. +5 −5 src/SocketImpl.h
  48. +5 −5 src/TlsSocketImpl.h
  49. +7 −3 src/Url.cpp
  50. +9 −12 src/ZipReaderImpl.cpp
  51. +34 −0 src/boost_compitability.cpp
  52. +29 −1 stop-containers.sh
  53. +10 −8 tests/functional/BasicTests.cpp
  54. +18 −15 tests/functional/CRUD_test.cpp
  55. +4 −5 tests/functional/ConnectionCacheTests.cpp
  56. +1 −1 tests/functional/ConnectionPoolInstancesTest.cpp
  57. +1 −1 tests/functional/CookieTests.cpp
  58. +2 −1 tests/functional/HttpsTest.cpp
  59. +5 −1 tests/functional/InsertSerializerTest.cpp
  60. +61 −32 tests/functional/ManyConnectionsTest.cpp
  61. +8 −7 tests/functional/OwnIoserviceTests.cpp
  62. +2 −2 tests/functional/ProxyTests.cpp
  63. +33 −30 tests/functional/ReadmeTests.cpp
  64. +1 −1 tests/functional/UploadTests.cpp
  65. +6 −2 tests/unit/CMakeLists.txt
  66. +334 −361 tests/unit/HttpReplyTests.cpp
  67. +13 −14 tests/unit/Iostream2JsonTests.cpp
  68. +45 −43 tests/unit/JsonSerializeTests.cpp
  69. +39 −21 tests/unit/UrlTests.cpp
107 changes: 107 additions & 0 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
name: CI

on:
push:
pull_request:
schedule:
- cron: '0 0 1 * *' # This line schedules the workflow to run at 00:00 on the first day of every month

defaults:
run:
shell: bash

jobs:
build:
runs-on: ${{ matrix.os }}
strategy:
matrix:
include:
- os: ubuntu-latest
compiler: gcc
- os: ubuntu-latest
compiler: clang
- os: windows-latest
compiler: msvc
- os: macos-latest
compiler:

steps:
- name: Checkout code
uses: actions/checkout@v4
with:
submodules: true

- name: Cache
uses: actions/cache@v4
with:
path: |
~/vcpkg
~/vcpkg_installed
${{ env.HOME }}/.cache/vcpkg/archives
${{ env.XDG_CACHE_HOME }}/vcpkg/archives
${{ env.LOCALAPPDATA }}\vcpkg\archives
${{ env.APPDATA }}\vcpkg\archives
key: ${{ runner.os }}-${{ matrix.compiler }}-${{ env.BUILD_TYPE }}-${{ hashFiles('**/CMakeLists.txt') }}-${{ hashFiles('./vcpkg.json')}}
restore-keys: |
${{ runner.os }}-${{ env.BUILD_TYPE }}-
- name: Setup Cpp
uses: aminya/setup-cpp@v1
with:
compiler: ${{ matrix.compiler }}
vcvarsall: ${{ contains(matrix.os, 'windows') }}
cmake: true
ninja: true
vcpkg: true
cppcheck: false

- name: Install compiler for Macos
if: startsWith(matrix.os, 'macos')
run: |
brew install llvm
- name: Prepare the PATH
run: |
if [[ "${{ runner.os }}" == "Windows" ]]; then
echo "$env:USERPROFILE\vcpkg" >> $GITHUB_PATH
echo "$env:USERPROFILE\ninja" >> $GITHUB_PATH
else
echo "$HOME/vcpkg" >> $GITHUB_PATH
echo "$HOME/ninja" >> $GITHUB_PATH
fi
- name: Install dependencies
run: |
cp -v ci/vcpkg/vcpkg.json .
vcpkg install
- name: Build project
run: |
pushd ~
if [ -d build ]; then
echo "Build dir exists"
ls -la build
else
mkdir -v build
fi
cd build
pwd
set -x
cmake -DVCPKG_INSTALLED_DIR=~/vcpkg_installed -DVCPKG_VERBOSE=ON -DRESTC_CPP_THREADED_CTX=ON -DCMAKE_BUILD_TYPE=Release -G "Ninja" -DCMAKE_TOOLCHAIN_FILE=~/vcpkg/scripts/buildsystems/vcpkg.cmake ${GITHUB_WORKSPACE}
cmake --build .
popd
continue-on-error: true

- name: Dump diagnostics
if: failure()
run: |
cd ~/build
echo "---------------------------------"
cat build.ninja
echo "---------------------------------"
- name: Run Unit Tests
run: |
pushd ~/build
ctest -R UNITTESTS . -C Release
popd
115 changes: 80 additions & 35 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,17 +1,11 @@
cmake_minimum_required(VERSION 3.0)

if (UNIX)
# Ninja creates too many problems while maintaining
# compatibility with old and new versions of Linux/Cmake
set (CMAKE_GENERATOR "Unix Makefiles" CACHE INTERNAL "" FORCE)
endif()
cmake_minimum_required(VERSION 3.10)

if (DEFINED ENV{RESTC_CPP_VERSION})
set(RESTC_CPP_VERSION $ENV{RESTC_CPP_VERSION})
endif()

if (NOT DEFINED RESTC_CPP_VERSION)
set(RESTC_CPP_VERSION 0.97.0)
set(RESTC_CPP_VERSION 1.0.0)
endif()

if(NOT DEFINED RESTC_BOOST_VERSION)
@@ -22,6 +16,18 @@ project (restc-cpp VERSION ${RESTC_CPP_VERSION})

message(STATUS "Building restc-cpp version ${PROJECT_VERSION}")

include(CheckCXXCompilerFlag)

if (MSVC)
# Thank you Microsoft. Its so nice of you to give us all these meaningful reasons to stay up all night.
check_cxx_compiler_flag("/std:c++20" COMPILER_SUPPORTS_CXX20)
check_cxx_compiler_flag("/std:c++17" COMPILER_SUPPORTS_CXX17)
add_compile_options(/Zc:__cplusplus)
else()
check_cxx_compiler_flag("-std=c++20" COMPILER_SUPPORTS_CXX20)
check_cxx_compiler_flag("-std=c++17" COMPILER_SUPPORTS_CXX17)
endif()

if (NOT DEFINED INSTALL_RAPIDJSON_HEADERS)
option(INSTALL_RAPIDJSON_HEADERS "Install rapidjson headers when make install is executed" ON)
endif()
@@ -32,6 +38,30 @@ option(RESTC_CPP_AUTORUN_UNIT_TESTS "Run Unit Tests automatically after build" O

option(RESTC_CPP_WITH_FUNCTIONALT_TESTS "Enable Functional Testing" ON)

option(RESTC_USE_LEGACY_BOOST_FIND "Use the old Boost find module" OFF)

set(GTEST_TAG "main" CACHE STRING "Gtest branch to use. Required on older Linux versions because newer gtest requure newer cmake!")
set(LOGFAULT_TAG "master" CACHE STRING "Logfault branch to use. Required on older Linux versions because newer gtest requure newer cmake!")

find_package(RapidJSON QUIET)

if (NOT RapidJSON_FOUND AND INSTALL_RAPIDJSON_HEADERS)
message(STATUS "Rapidjson not found. Adding it as an external project.")
set(RESTC_CPP_EXTERNAL_DEPS ${RESTC_CPP_EXTERNAL_DEPS} externalRapidJson)
set(restc_cpp_add_rapidjson ON)
endif()

if (RESTC_CPP_LOG_WITH_LOGFAULT)
find_path(LOGFAULT logfault/logfault.h)
if (LOGFAULT)
message ("Using existing logfault")
else()
message ("Embedding logfault header only library")
set(RESTC_CPP_EXTERNAL_DEPS ${RESTC_CPP_EXTERNAL_DEPS} externalLogfault)
set(restc_cpp_add_logfault ON)
endif()
endif()

include(cmake_scripts/external-projects.cmake)

if (EXISTS ${CMAKE_BINARY_DIR}/conanbuildinfo.cmake)
@@ -68,7 +98,26 @@ option(RESTC_CPP_LOG_JSON_SERIALIZATION "Enable trace logging for json serializa

option(RESTC_CPP_WITH_ZLIB "Use zlib" ON)

option(RESTC_CPP_USE_CPP17 "Use the C++17 standard" ON)
option(RESTC_CPP_USE_CPP14 "Use the C++14 standard" OFF)

option(RESTC_CPP_USE_CPP17 "Use the C++17 standard" OFF)

option(RESTC_CPP_USE_CPP20 "Use the C++20 standard" OFF)

if (NOT RESTC_CPP_USE_CPP14 AND NOT RESTC_CPP_USE_CPP17 AND NOT RESTC_CPP_USE_CPP20)
# Check if GCC 12 is being used and default it to C++17
# https://www.mail-archive.com/debian-bugs-dist@lists.debian.org/msg1887728.html
if (CMAKE_CXX_COMPILER_ID STREQUAL "GNU" AND CMAKE_CXX_COMPILER_VERSION VERSION_LESS 13.0 AND CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 12.0)
message(STATUS "GCC 12 detected, defaulting to C++17")
set(RESTC_CPP_USE_CPP17 ON)
elseif (COMPILER_SUPPORTS_CXX20)
set(RESTC_CPP_USE_CPP20 ON)
elseif (COMPILER_SUPPORTS_CXX17)
set(RESTC_CPP_USE_CPP17 ON)
else ()
set(RESTC_CPP_USE_CPP14 ON)
endif()
endif()

option(RESTC_CPP_THREADED_CTX "Allow asio contextx with multiple therads. Enables thread-safe internal access." OFF)

@@ -139,7 +188,10 @@ endif()
message(STATUS "Using ${CMAKE_CXX_COMPILER}")

macro(SET_CPP_STANDARD target)
if (RESTC_CPP_USE_CPP17)
if (RESTC_CPP_USE_CPP20)
message(STATUS "Using C++ 20 for ${target}")
set_property(TARGET ${target} PROPERTY CXX_STANDARD 20)
elseif (RESTC_CPP_USE_CPP17)
message(STATUS "Using C++ 17 for ${target}")
set_property(TARGET ${target} PROPERTY CXX_STANDARD 17)
else()
@@ -168,6 +220,7 @@ set(ACTUAL_SOURCES
src/RequestBodyStringImpl.cpp
src/RequestBodyFileImpl.cpp
src/url_encode.cpp
src/boost_compitability.cpp
${LOGGING_SRC}
)

@@ -179,14 +232,7 @@ if (RESTC_CPP_WITH_ZLIB)
set(ACTUAL_SOURCES ${ACTUAL_SOURCES} src/ZipReaderImpl.cpp)
endif()

if (WIN32)
include(cmake_scripts/pch.cmake)
ADD_MSVC_PRECOMPILED_HEADER(restc-cpp/restc-cpp.h src/pch.cpp ACTUAL_SOURCES)
add_definitions(-DWAR_PCH)
set(SOURCES ${ACTUAL_SOURCES} src/pch.cpp ${HEADERS} ${RESFILES})
else()
set(SOURCES ${ACTUAL_SOURCES})
endif()
set(SOURCES ${ACTUAL_SOURCES})

add_library(${PROJECT_NAME} ${SOURCES})
set_target_properties(${PROJECT_NAME} PROPERTIES DEBUG_OUTPUT_NAME restc-cppD)
@@ -199,21 +245,21 @@ target_include_directories(${PROJECT_NAME}

SET_CPP_STANDARD(${PROJECT_NAME})

if (RESTC_CPP_LOG_WITH_LOGFAULT)
find_path(LOGFAULT logfault/logfault.h)
if (LOGFAULT)
message ("Using existing logfault")
else()
message ("Embedding logfault header only library")
set(RESTC_CPP_EXTERNAL_DEPS ${RESTC_CPP_EXTERNAL_DEPS} externalLogfault)
endif()
if(NOT "${RESTC_CPP_EXTERNAL_DEPS}" STREQUAL "")
add_dependencies(${PROJECT_NAME} ${RESTC_CPP_EXTERNAL_DEPS})
endif()

add_dependencies(${PROJECT_NAME} externalRapidJson ${RESTC_CPP_EXTERNAL_DEPS})

if (NOT EMBEDDED_RESTC_CPP)
#set(Boost_USE_MULTITHREADED ON)
find_package(Boost ${RESTC_BOOST_VERSION} REQUIRED COMPONENTS

if (RESTC_USE_LEGACY_BOOST_FIND)
unset(restc_cpp_boost_find_config)
message("Using legacy Boost find config")
elseif(CMAKE_VERSION VERSION_GREATER "3.28")
set(restc_cpp_boost_find_config CONFIG)
message("Using new Boost find config")
endif()

find_package(Boost ${RESTC_BOOST_VERSION} REQUIRED ${restc_cpp_boost_find_config} COMPONENTS
system
program_options
filesystem
@@ -223,6 +269,9 @@ if (NOT EMBEDDED_RESTC_CPP)
chrono
${BOOST_LOG_DEP}
)

message(STATUS "Boost version found: ${Boost_VERSION}")

target_include_directories(${PROJECT_NAME} PUBLIC ${Boost_INCLUDE_DIRS})
target_link_libraries(${PROJECT_NAME} PUBLIC ${Boost_LIBRARIES})
target_compile_definitions(${PROJECT_NAME} PUBLIC -DBOOST_COROUTINE_NO_DEPRECATION_WARNING=1)
@@ -255,16 +304,12 @@ if (NOT EMBEDDED_RESTC_CPP)
link_directories(${Boost_LIBRARY_DIRS})
endif()

include(cmake_scripts/pch.cmake)

set_property(TARGET PROPERTY CXX_STANDARD 17)

if(WIN32)
add_definitions(-D_WIN32_WINNT=0x0600)
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} /SAFESEH:NO")
endif()

set(LIBRARY_OUTPUT_PATH ${PROJECT_SOURCE_DIR}/lib CACHE PATH "Destination location")
set(LIBRARY_OUTPUT_PATH ${CMAKE_BINARY_DIR}/lib CACHE PATH "Destination location")
link_directories(${LIBRARY_OUTPUT_PATH})

include(cmake_scripts/doxygen.cmake)
47 changes: 35 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
[![CI](https://github.com/jgaa/restc-cpp/actions/workflows/ci.yaml/badge.svg)](https://github.com/jgaa/restc-cpp/actions/workflows/ci.yaml)

# Introduction to the restc-cpp C++ library
<i>The magic that takes the pain out of accessing JSON API's from C++ </i>

@@ -73,7 +75,7 @@ that experience. So I spent a few weeks creating my own HTTP Client library
using boost::asio with JSON serialization/deserialization.

# Dependencies
Restc-cpp depends on C++14 with its standard libraries and:
Restc-cpp depends on C++14 (or newer) with its standard libraries and:
- boost
- rapidjson (CMake will download and install rapidjson for the project)
- gtest (CMake will download and install gtest for the project if it is not installed)
@@ -251,33 +253,54 @@ Please refer to the [tutorial](doc/Tutorial.md) for more examples.
# Current Status
The project has been in public BETA since April 11th 2017.

# Supported compilers
These are the compilers that are being tested before anything is merged to the master branch.

- g++ from 8 to 14
- clang (current)
- msvc (current)
- Apple clang (current)

# Supported C++ standards
These are the C++ versions that are are being tested before anything is merged to the master branch.

- C++14
- C++17
- C++20

# Supported operating systems
These are the operating systems where my Continues Integration (Jenkins) servers currently compiles the project and run all the tests:

- Debian Testing
- Debian Bullseye (Stable)
- Debian Bookworm
- Debian Bullseye
- Debian Buster
- Debian Stretch
- Windows 10 / Microsoft Visual Studio 2019, Community version using vcpkg for dependencies
- Ubuntu Xenial (LTS)
- Windows 10 / Microsoft Visual Studio 2022, Community version using vcpkg for dependencies
- Ubuntu Noble (LTS)
- Ubuntu Jammy (LTS)
- Ubuntu Bionic (LTS)

Support for MacOS has been removed after Apples announcement that their love for privacy was just
a marketing gimmick.

Fedora is currently disabled in my CI because of failures to start their Docker containers. (Work in progress). Ubuntu Jammy don't work in docker with my Jenkins CI pipeline, so I have no reliable way to test it. Windows 11 cannot be run on my KVM /QEMU system, because it don't support "secure" boot, so I have no way to test it.
- Fedora (latest)
- MacOS (latest)

The Jenkins setup is [here](ci/jenkins).

I currently use my own CI infrastructure running on my own hardware. I use Jenkins on a VM with Debian Bullseye, and three slaves for Docker on Linux VM's, one slave running on a VM with Microsoft Windows 10 Pro. Using Docker to build with different Linux distributions gives me flexibility. It also immediately catches mistakes that break the build or test(s) on a specific Linux distribution or platform. Using my own infrastructure improves the security, as I don't share any credentials with 3rd party services or allow external access into my LAN.
I currently use my own CI infrastructure running on my own hardware. I use Jenkins on a VM with Debian Bookworm, and three slaves for Docker on Linux VM's, one slave running on a VM with Microsoft Windows 10 Pro. Using Docker to build with different Linux distributions gives me flexibility. It also immediately catches mistakes that break the build or test(s) on a specific Linux distribution or platform. Using my own infrastructure improves the security, as I don't share any credentials with 3rd party services or allow external access into my LAN.

Github Actions can not compile for various Linux variants (at least not on the free plan for Open Source projects),
and it can not run multiple docker-containers (or even *any* containers for Windows or MacOS builds) to allow integration testing.
I have configured it for this repository anyway, because it's automation setup is different than the Jenkins setup,
which have helped identifying some issues with the projects cmake files.

# Blog-posts about the project:
- [About version 0.90](https://lastviking.eu/restc_cpp_90.html)
- [restc-cpp tags on The Last Viking's Nest](https://lastviking.eu/_tags/restc-cpp.html)

# Similar projects
- [Boost.Beast](https://github.com/boostorg/beast) by Vinnie Falco. When you like to write many lines of code...
- [RESTinCurl](https://github.com/jgaa/RESTinCurl) by me. Aimed at mobile applications, IoT and projects that already link with libcurl.
- [Boost.Beast](https://github.com/boostorg/beast) by Vinnie Falco. When you like to write many lines of code...

**Json serialization only**
- [Boost.Json](https://www.boost.org/doc/libs/1_83_0/libs/json/doc/html/index.html)
- [JSON for Modern C++](https://nlohmann.github.io/json/) by Niels Lohmann. My favorite json library, when I need to more than just static serialization.
- [json11 - tiny JSON library for C++11, providing JSON parsing and serialization](https://github.com/dropbox/json11)

Loading