DRLTT SDK

Software Development Kit (SDK) for deploying DRLTT into a real-time system, CPU-only and runtime efficient.

Interfaces

The Interface of C++ SDK: sdk/drltt-sdk/trajectory_tracker/trajectory_tracker.h

The Interface of Python SDK: sdk/assets/exported-python-sdk/trajectory_tracker.py

Compilation and Exporting

This project employs cmake as build system.. The compilation is recommended to be done within a Docker container.

Build Docker Image

Firstly, build an image named drltt-sdk for compilation with the provided Dockerfile.

docker image build --tag drltt-sdk:dev - < ./Dockerfile

Tip: To remove unused images/cached, run:

docker system prune

Tip 2: To save the Docker image for transferring and save time:

docker image save drltt-sdk:dev -o ./drltt-sdk.image

docker image load -i ./drltt-sdk.image

Tip 3: For network environments within Mainland China, you may consider using a domestic apt source to accelerate this process by appending the following part to the ./Dockerfile:

# Example using TUNA apt source
ARG APT_SOURCE_LIST=/etc/apt/sources.list
RUN \
    mv ${APT_SOURCE_LIST} ${APT_SOURCE_LIST}.bak && \
    touch ${APT_SOURCE_LIST} && \
    printf "deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ jammy main restricted universe multiverse" >> ${APT_SOURCE_LIST} && \
    printf "deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ jammy-updates main restricted universe multiverse" >> ${APT_SOURCE_LIST} && \
    printf "deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ jammy-backports main restricted universe multiverse" >> ${APT_SOURCE_LIST} && \
    printf "deb http://security.ubuntu.com/ubuntu/ jammy-security main restricted universe multiverse" >> ${APT_SOURCE_LIST} && \
    cat ${APT_SOURCE_LIST}

Debugging the building process

./compile-in-docker.sh can be used to debug the building process. Pass interaction argument to enter the container instead of launching the building process directly.

./compile-in-docker.sh interactive

Furthermore, pass nonsudo argument to avoid entering SUDO account.

./compile-in-docker.sh interactive nonsudo

Compile Source and Export SDK Shared Library within Docker Container

Secondly, launch compilation and exporting by running ./compile-in-docker.sh.

#!/bin/bash

build_dir=./build
project_name=drltt-sdk
image_name=${project_name}:dev

docker_source_dir=/${project_name}
docker_repo_work_dir=/drltt-work_dir
docker_checkpoint_dir=${docker_repo_work_dir}/track-test/checkpoint
docker_usr_lib_dir=/usr/local/lib
docker_proto_gen_dir=${docker_source_dir}/proto_gen

if [[ -v HOST_LIBTORCH_DIR ]];then
    docker_libtorch_dir=/libtorch-host
    mount_host_libtorch_in_docker="-v ${HOST_LIBTORCH_DIR}:${docker_libtorch_dir}:ro"
    echo "Using libtorch mounted from host: ${HOST_LIBTORCH_DIR}"
else
    docker_libtorch_dir=/libtorch
    mount_host_libtorch_in_docker=""
    echo "Using libtorch preinstalled in docker image"
fi

if [[ -v HOST_PYTORCH_DIR ]];then
    docker_pytorch_dir=/pytorch-host
    mount_pytorch_in_docker="-v $PYTORCH_DIR:/${docker_pytorch_dir}:rw"
else
    docker_pytorch_dir=/pytorch
    mount_pytorch_in_docker=""
fi

# TODO: reorganize mouted path

nonsudo_user_arg=" --user $(id -u):$(id -g) "
nonsudo_user_source_cmd=" source /work_dir/.bashrc "

if [[ $1 == "interactive" ]]; then
    docker_arg_suffix=" -it ${image_name} "
    docker_container_cmd=" bash "
    if [[ $2 == "nonsudo" ]]; then
        docker_arg_suffix="${nonsudo_user_arg} ${docker_arg_suffix}"
        docker_container_cmd=" ${nonsudo_user_source_cmd} && bash "
    fi
else
    docker_arg_suffix="${nonsudo_user_arg} ${image_name} "
    docker_container_cmd=" ${nonsudo_user_source_cmd} && cd ${docker_source_dir} && bash ./compile-source.sh"
    if [[ ! $1 == "test" ]]; then
        docker_container_cmd="${docker_container_cmd} && bash ./export-py-sdk.sh"
    fi
fi

docker run --name drltt-sdk --entrypoint bash -e "ACCEPT_EULA=Y" --rm --network=host \
    -e "PRIVACY_CONSENT=Y" \
    -e "PROJECT_NAME=${project_name}" \
    -e "SOURCE_DIR=${docker_source_dir}" \
    -e "PROTO_GEN_DIR=${docker_proto_gen_dir}" \
    -e "BUILD_DIR=${build_dir}" \
    -e "REPO_WORK_DIR=${docker_repo_work_dir}" \
    -e "CHECKPOINT_DIR=${docker_checkpoint_dir}" \
    -e "USR_LIB_DIR=${docker_usr_lib_dir}" \
    -e "LIBTORCH_DIR=${docker_libtorch_dir}" \
    -e "PYTORCH_DIR=${docker_pytorch_dir}" \
    -v $PWD:/${docker_source_dir}:rw \
    -v $PWD/../common/proto:/proto:rw \
    -v $PWD/../work_dir:${docker_repo_work_dir}:ro \
    ${mount_pytorch_in_docker} \
    ${mount_host_libtorch_in_docker} \
    ${docker_arg_suffix} \
    -c "${docker_container_cmd}"

To use LibTorch on the host during the compilation phase, please pass an environment variable HOST_LIBTORCH_DIR to ./compile-in-docker.sh:

HOST_LIBTORCH_DIR=/path/to/libtorch/on/host ./compile-in-docker.sh

Compiling and exporting inside the Docker container under ./sdk/build

Inside the container, it will first compile the Protobuf (this is important for Protobuf to be included successfully) and then compile source files.

#!/bin/bash

# NOTE: current directory is `./sdk/build`

# shared libraries
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$USR_LIB_DIR

# TODO figure out root cause and resolve it in more formal way
# https://stackoverflow.com/questions/19901934/libpthread-so-0-error-adding-symbols-dso-missing-from-command-line
export LDFLAGS="-Wl,--copy-dt-needed-entries"

# Check version and location of Protobuf compiler
echo "Using protoc $(protoc --version) at $(which protoc)"

# Clear exisiting compiled files
rm -rf ${BUILD_DIR}
mkdir -p ${BUILD_DIR}
rm -rf ${PROTO_GEN_DIR}
mkdir -p ${PROTO_GEN_DIR}

# Configure and build
pushd ${BUILD_DIR}
    cmake .. \
        -DBUILD_TESTS=ON \
        -DMACRO_CHECKPOINT_DIR=${CHECKPOINT_DIR} \
        -DLIBTORCH_DIR=${LIBTORCH_DIR} \
        && make -j$(nproc --all) 2>&1 | tee ./build.log
    ctest -VV --rerun-failed --output-on-failure 2>&1 | tee ./test.log
popd

Secondly, an optional exporting step will export a standalone Python SDK library backed with the compiled C++ SDK, along with all shared dependent libraries.

#!/bin/bash

# NOTE: current directory is `./sdk/build`

echo "Exporting Python SDK at ${export_dir}..."

project_name=${PROJECT_NAME}
package_name=$(echo ${project_name}-py | sed -r 's/-/_/g')
export_dir=${BUILD_DIR}/${package_name}
mkdir -p $export_dir

# export dependency shared library.
#   TODO: consider a more elegant way, like packaging
libtorch_lib_dir=${LIBTORCH_DIR}/lib
cp -r ${USR_LIB_DIR} $export_dir/
echo "User lib size: $(du -sh $USR_LIB_DIR)"
if [[ ! -v PY_SDK_NO_LIBTORCH_EXPORTED ]];then
  cp -r ${libtorch_lib_dir} $export_dir/
  echo "libtorch lib size: $(du -sh $libtorch_lib_dir)"
fi
echo "Total exported lib size: $(du -sh $export_dir/lib)"

# export sdk shared library
sdk_so=$(ls ${BUILD_DIR}/${project_name}/trajectory_tracker/trajectory_tracker_*.so|head -n 1)
cp $sdk_so $export_dir/
pushd $export_dir/
  ln -sf $(basename $sdk_so) export_symbols.so
popd

# export assets
cp -r ${CHECKPOINT_DIR} $export_dir/
cp -r assets/exported-python-sdk/* $export_dir/
cp -r /proto/proto_gen_py $export_dir/

# package into tarball
# TODO: move to a more formal packaging way
pushd ${BUILD_DIR}
  tar -czf ./${package_name}.tar.gz ${package_name}/
  echo "library packed: ${package_name}.tar.gz"
popd

The standalone library is under ./sdk/build/drltt_sdk_py.tar.gz. Currently, the library is for Python 3.8 (version-specific as ABI changes across versions). TOOD: Support for multiple Python versions is coming soon.

Tree structure within docker container

/
├── usr
│   └── local
│       ├── bin
│       └── lib
├── proto -> $PROJECT_ROOT/common/proto:/proto      # protobuf source
├── drltt-sdk -> $PROJECT_ROOT/sdk/drltt-sdk        # `source_dir`
│   └── proto_gen                                   # generated protobuf
└── work_dir

build folder structure

Building results are exported to the build folder which has the following structure:

/build
├── ...
├── proto_def
├── drltt-sdk             # compiled libraries and executables
│   ├── common
│   ├── inference
│   ├── trajectory_tracker
│   └── trajectory_tracker
├── lib                   # exported shared library for running
├── drltt_sdk_py          # exported standalone python sdk library
├── drltt_sdk_py.tar.gz   # packed standalone python sdk library
└── ...

Run sample program

TODO: An executable sample program is coming soon.

If you prefer to use shared libraries on your host machine, please prepend your shared libraries’ path to LD_LIBRARY_PATH.

Static Linking with LibTorch

To ensure static linking with LibTorch, you may need to clone the PyTorch’s source code and compile a static version from the source:

PYTORCH_DIR=path/to/pytorch ./compile-in-docker.sh interactive
# in docker container
cd $PYTORCH_DIR && ./scripts/build_mobile.sh

References:

• https://stackoverflow.com/questions/60629537/cmake-linking-static-library-pytorch-cannot-find-its-internal-functions-during

• https://github.com/pytorch/pytorch/blob/main/scripts/build_mobile.sh

• https://github.com/szymonmaszke/torchlambda/blob/c70d3ea956972965e2bfe7a33213f9f6ba814972/src/dependencies/torch.sh

Deployment

NOTE: Currently, Python SDK may not be compatible with the user’s PyTorch installation due to incompatibility of shared library unless the user export it on the host machine. Plan to be resolved in the future.

Unpackage the exported tarball and set LD_LIBRARY_PATH manually (effectively modifying it during runtime is not possible):

tar xvzf drltt_sdk_py.tar.gz -C ./
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:./drltt_sdk_py/lib

Reference: https://man7.org/linux/man-pages/man8/ld.so.8.html

Import drltt_sdk_py. As no depedency required except for Python=3.8, you can run the trajectory tracking directly:

from drltt_sdk_py import TrajectoryTracker

tracker = TrajectoryTracker()
reference_line = [(0.1 * 5.0 * step_index, 0.1 * 4.0 * step_index) for step_index in range(60)]
tracked_states, tracked_actions = tracker.track_traference_line(reference_line)

To check the numeric precision, run the following test:

cd drltt_sdk_py && ./check.sh

Development

Code format

This project uses clang-format to format CXX code according to Google C/C++ Code Style settings.

To launch code-format, run bash format-code.sh:

#!/bin/bash

echo "FORMATTING CPP CODE..."
find . -regex '.*\.\(cpp\|hpp\|cu\|cuh\|c\|h\)' \
    -not -path "./build/*" \
    -not -path "./proto_gen/*" \
    -exec clang-format \
    --verbose --style=file -i {} \;

To customize your own clang-format config file, run:

clang-format -style=llvm -dump-config > .clang-format

Debugging

Following global objects are useful for achieving runtime result consistency on the Python side and the C++ side:

• python: common.io.GLOBAL_DEBUG_INFO

• cpp: drltt::global_debug_info

For example, to compare runtime values of rotation_radius_inv, you may add two lines of code as follows before running ./test/test-cpp.sh fast to launch the debugging process:

diff --git a/sdk/drltt-sdk/dynamics_models/bicycle_model.cpp b/sdk/drltt-sdk/dynamics_models/bicycle_model.cpp
index b694343..bcf01e4 100644
--- a/sdk/drltt-sdk/dynamics_models/bicycle_model.cpp
+++ b/sdk/drltt-sdk/dynamics_models/bicycle_model.cpp
@@ -96,6 +96,8 @@ bool BicycleModel::ComputeRotationRelatedVariables(
   *rotation_radius_inv =
       std::sin(*omega) / _hyper_parameter.bicycle_model().rearwheel_to_cog();

+  global_debug_info.add_data(*rotation_radius_inv);
+
   return true;
 }

diff --git a/simulator/dynamics_models/bicycle_model.py b/simulator/dynamics_models/bicycle_model.py
index a2df076..c93e8fe 100644
--- a/simulator/dynamics_models/bicycle_model.py
+++ b/simulator/dynamics_models/bicycle_model.py
@@ -180,6 +180,8 @@ class BicycleModel(BaseDynamicsModel):
         omega = normalize_angle(omega)
         rotation_radius_inv = np.sin(omega) / hyper_parameter.rearwheel_to_cog

+        from common import GLOBAL_DEBUG_INFO;GLOBAL_DEBUG_INFO.data.append(rotation_radius_inv)
+
         return omega, rotation_radius_inv

     @property

Check out log file to check the data. gt_data (ground-truth) denotes value on the Python side and rt_data (runtime) denotes value on the C++ side.

5: Test case: 323, Step: 28
5:     gt_data: 0.000584156, rt_data: 0.000584146
5: Test case: 323, Step: 29
5:     gt_data: -8.38374e-05, rt_data: -8.38471e-05
5: Test case: 323, Step: 30
5:     gt_data: -0.00092932, rt_data: -0.00092933
5: Test case: 323, Step: 31
5:     gt_data: -0.00189424, rt_data: -0.00189425

References:

• https://clang.llvm.org/get_started.html

• https://www.kernel.org/doc/html/next/process/clang-format.html

• https://leimao.github.io/blog/Clang-Format-Quick-Tutorial/