728x90
출처
- Gemma 4 E4B | Jetson AI Lab
- Models | Jetson AI Lab
- TensorRT-LLM/examples/models/core/qwen/README.md at main · NVIDIA/TensorRT-LLM
CUDA 버전 확인
bluesanta@localhost:~$ cat /proc/device-tree/model
NVIDIA Jetson Orin NX Engineering Reference Developer Kit
bluesanta@localhost:~$ uname -a
Linux localhost.localdomain 6.8.12-1021-tegra #1 SMP PREEMPT Thu Aug 6 21:56:04 PDT 2026 aarch64 aarch64 aarch64 GNU/Linux
bluesanta@localhost:~$ cat /etc/nv_tegra_release
## R39 (release), REVISION: 2.1, GCID: 46758480, BOARD: generic, EABI: aarch64, DATE: Fri Aug 7 05:54:22 AM UTC 2026
## KERNEL_VARIANT: oot
TARGET_USERSPACE_LIB_DIR=nvidia
TARGET_USERSPACE_LIB_DIR_PATH=usr/lib/aarch64-linux-gnu/nvidia
bluesanta@localhost:~$ nvidia-smi --query-gpu=name,compute_cap,driver_version --format=csv
name, compute_cap, driver_version
Orin (nvgpu), 8.7, 595.78
bluesanta@localhost:~$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2026 NVIDIA Corporation
Built on Fri_May_08_10:52:52_AM_PDT_2026
Cuda compilation tools, release 13.2, V13.2.86
Build cuda_13.2.r13.2/compiler.37953736_0파이썬 버전 확인
bluesanta@localhost:~$ python --version
Python 3.12.3빌드 도구 설치
bluesanta@localhost:~$ sudo apt upgrade
bluesanta@localhost:~$ sudo apt install -y cmake ninja-build gcc g++ git build-essential git cmake ninja-build libopenblas-dev libopenmpi-dev openmpi-bin libatlas-base-dev libprotobuf-dev protobuf-compiler libssl-dev zlib1g-dev libffi-dev python3-pip libopenblas-dev ccache git-lfs libjpeg-dev libpng-dev libtiff-dev원격 데스크톱(xrdp) 설치
xrdp 패키지 설치 및 기본 활성화
서버 터미널(SSH)에서 아래 명령어를 실행하여 xrdp를 설치하고 서비스 등록을 진행합니다.
$ # 패키지 목록 업데이트 및 xrdp 설치
$ sudo apt update
$ sudo apt install -y xrdp
$
$ # 부팅 시 자동 시작 등록 및 서비스 시작
$ sudo systemctl enable --now xrdp
$
$ # SSL 인증서 권한 추가 (중요: xrdp 계정이 ssl-cert 키를 읽을 수 있어야 함)
$ sudo adduser xrdp ssl-cert
$
$ # 서비스 재시작
$ sudo systemctl restart xrdp데스크톱 환경 세션 파일(~/.xsession) 생성
xrdp 접속 시 띄울 윈도우 매니저(세션)를 명시적으로 설정해 주어야 검은 화면에서 튕기는 현상을 막을 수 있습니다.
$ echo "gnome-session" > ~/.xsession
$ chmod +x ~/.xsessionGNOME 확장(Extensions) 관리 유틸리티 설치
bluesanta@localhost:~$ sudo apt install -y gnome-shell-extension-manager gnome-tweaks오류 확인
bluesanta@localhost:~$ cat .xsession-errors
Xsession: X session started for bluesanta at Mon Sep 28 05:13:10 PM KST 2026
dbus-update-activation-environment: setting DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/2002/bus
dbus-update-activation-environment: setting DISPLAY=:10.0
localuser:bluesanta being added to access control list
/etc/X11/Xsession: 85: /home/bluesanta/.xsessionrc: Syntax error: "(" unexpected (expecting "fi")~/.xsessionrc를 POSIX sh 호환 문법으로 수정
48번 줄 수정 (== → =)
# 기존
if [ "$(lsb_release -cs)" == "bionic" ]; then
# 변경
if [ "$(lsb_release -cs)" = "bionic" ]; then85~91번 줄 수정 (배열 문법 제거)
# 기존
if [ -e "/usr/bin/gnome-shell" ]; then
remove_apps=("thunderbird" "rhythmbox" "libreoffice-writer")
favorite_apps="$(gsettings get org.gnome.shell favorite-apps)"
for app in "${remove_apps[@]}"; do
favorite_apps="$(echo "${favorite_apps}" | \
sed -e "s/'${app}.desktop', //g" -e "s/, '${app}.desktop' / /g")"
done
gsettings set org.gnome.shell favorite-apps "${favorite_apps}"
fi# 변경
if [ -e "/usr/bin/gnome-shell" ]; then
favorite_apps="$(gsettings get org.gnome.shell favorite-apps)"
for app in thunderbird rhythmbox libreoffice-writer; do
favorite_apps="$(echo "${favorite_apps}" | \
sed -e "s/'${app}.desktop', //g" -e "s/, '${app}.desktop' / /g")"
done
gsettings set org.gnome.shell favorite-apps "${favorite_apps}"
fi도커 로그인
bluesanta@localhost:~$ docker login
USING WEB-BASED LOGIN
i Info → To sign in with credentials on the command line, use 'docker login -u '
Your one-time device confirmation code is: XXXX-XXXX
Press ENTER to open your browser or submit your device code here: https://login.docker.com/activate
Waiting for authentication in the browser…
WARNING! Your credentials are stored unencrypted in '/home/bluesanta/.docker/config.json'.
Configure a credential helper to remove this warning. See
https://docs.docker.com/go/credential-store/
Login Succeeded도커 로그인
bluesanta@localhost:~$ sudo docker run -it --rm --pull always --runtime=nvidia --network host -v ~/.cache/huggingface:/root/.cache/huggingface -v ~/.cache/vllm:/root/.cache/vllm vllm/vllm-openai:latest google/gemma-4-E4B-it-qat-w4a16-ct --max-model-len 8192 --gpu-memory-utilization 0.7 --reasoning-parser gemma4 --enable-auto-tool-choice --tool-call-parser gemma4 --default-chat-template-kwargs '{"enable_thinking":true}' --speculative-config '{"method":"mtp","model":"google/gemma-4-E4B-it-assistant","num_speculative_tokens":3}'
latest: Pulling from vllm/vllm-openai도커 로그인
bluesanta@localhost:~/llm/models$ sudo docker run -it --rm --pull always \
--runtime=nvidia --network host \
ghcr.io/nvidia-ai-iot/vllm:latest-jetson-orin \
vllm serve /home/bluesanta/llm/models/gemma-3-12b-it-quantized.w4a16 --gpu-memory-utilization 0.8
latest-jetson-orin: Pulling from nvidia-ai-iot/vllm
728x90