docs(07-03): Deployment testing plan and documentation
This commit is contained in:
@@ -0,0 +1,292 @@
|
||||
# Gravl Deployment Guide
|
||||
|
||||
This guide covers how to deploy the Gravl application, verify deployments, and troubleshoot common issues.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Docker and Docker Compose installed
|
||||
- Git repository with remote configured
|
||||
- Access to `/workspace/gravl` directory
|
||||
- Backend API listening on `http://localhost:3001/api/health`
|
||||
|
||||
## Deployment Script
|
||||
|
||||
### Running a Deployment
|
||||
|
||||
```bash
|
||||
cd /workspace/gravl
|
||||
scripts/deploy.sh
|
||||
```
|
||||
|
||||
### What It Does
|
||||
|
||||
1. **Pulls latest code:** `git pull`
|
||||
2. **Captures build metadata:**
|
||||
- Git commit SHA
|
||||
- Build timestamp
|
||||
3. **Builds fresh images:** `docker compose build --no-cache`
|
||||
- `--no-cache` ensures all layers are rebuilt (prevents stale assets)
|
||||
4. **Restarts containers:** `docker compose up -d --force-recreate`
|
||||
5. **Health check:** Polls `/api/health` for up to 60 seconds
|
||||
6. **Logs deployment:** Records all steps to `logs/deploy.log`
|
||||
|
||||
### Output Example
|
||||
|
||||
```
|
||||
[2026-03-03 18:30:00] === Deploy started ===
|
||||
[2026-03-03 18:30:01] Pulling latest code...
|
||||
[2026-03-03 18:30:05] Commit: 53f4df6 | Date: 2026-03-03T18:30:00Z
|
||||
[2026-03-03 18:30:06] Building images (--no-cache)...
|
||||
[2026-03-03 18:30:45] Starting containers...
|
||||
[2026-03-03 18:30:50] Health check...
|
||||
[2026-03-03 18:30:55] Backend healthy
|
||||
[2026-03-03 18:30:56] === Deploy complete: 53f4df6 ===
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Checking Deployment Status
|
||||
|
||||
### Build Status Check
|
||||
|
||||
```bash
|
||||
cd /workspace/gravl
|
||||
scripts/build-check.sh
|
||||
```
|
||||
|
||||
### Output Example
|
||||
|
||||
```
|
||||
Local HEAD: 53f4df6 (53f4df6f8a5c4d2e1f0a9b8c7d6e5f4a3b2c1d0)
|
||||
|
||||
[gravl-backend] Built: 53f4df6 on 2026-03-03T18:30:00Z
|
||||
[gravl-backend] OK: up to date
|
||||
|
||||
[gravl-frontend] Built: 53f4df6 on 2026-03-03T18:30:00Z
|
||||
[gravl-frontend] OK: up to date
|
||||
```
|
||||
|
||||
### What the Check Tells You
|
||||
|
||||
- **OK: up to date** — Containers match the local git commit (everything is current)
|
||||
- **STALE: container is behind local code** — Code has changed but containers haven't been redeployed yet
|
||||
- **WARNING: no build label found** — Container is old (pre-07-02) and lacks build tracking labels
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Health Check Failures
|
||||
|
||||
**Symptom:** Deployment fails with "ERROR: Health check failed after 60s"
|
||||
|
||||
**Possible Causes & Solutions:**
|
||||
|
||||
| Cause | Solution |
|
||||
|-------|----------|
|
||||
| Backend not starting | Check logs: `docker compose logs gravl-backend` |
|
||||
| Health endpoint not implemented | Implement `GET /api/health` in backend (returns `200 OK`) |
|
||||
| Network issues | Verify network: `docker network inspect gravl` or restart: `docker compose restart` |
|
||||
| Port already in use | Check: `lsof -i :3001` and kill the process or change port |
|
||||
| Insufficient resources | Free disk space: `df -h` or reduce image size |
|
||||
|
||||
**Manual Restart:**
|
||||
```bash
|
||||
docker compose restart gravl-backend
|
||||
# Wait a few seconds
|
||||
curl -sf http://localhost:3001/api/health
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Stale Containers
|
||||
|
||||
**Symptom:** `build-check.sh` shows "STALE: container is behind local code"
|
||||
|
||||
**Cause:** Code has been updated but containers haven't been redeployed.
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
scripts/deploy.sh
|
||||
scripts/build-check.sh # Should now show OK
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Missing Docker Labels
|
||||
|
||||
**Symptom:** `build-check.sh` shows "WARNING: no build label found"
|
||||
|
||||
**Cause:** Containers were built before phase 07-02 (before labels were added).
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
scripts/deploy.sh # Rebuilds with labels
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Deployment Hangs
|
||||
|
||||
**Symptom:** `scripts/deploy.sh` doesn't complete or appears stuck.
|
||||
|
||||
**Possible Causes & Solutions:**
|
||||
|
||||
| Symptom | Solution |
|
||||
|---------|----------|
|
||||
| Stuck at "Building images" | Docker build is slow. Check: `docker builder prune` to free cache |
|
||||
| Stuck at "Health check" | Backend not responding. Try: `docker compose logs` to see errors |
|
||||
| Git pull conflicts | Resolve conflicts manually: `cd /workspace/gravl && git status` |
|
||||
|
||||
**Force Stop:**
|
||||
```bash
|
||||
# Kill the deploy script
|
||||
pkill -f scripts/deploy.sh
|
||||
|
||||
# Manually check status
|
||||
docker compose ps
|
||||
docker compose logs
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Rollback Procedures
|
||||
|
||||
### Quick Rollback
|
||||
|
||||
If the current deployment is broken:
|
||||
|
||||
```bash
|
||||
# Revert to previous commit
|
||||
git reset --hard HEAD~1
|
||||
|
||||
# Redeploy
|
||||
scripts/deploy.sh
|
||||
|
||||
# Verify
|
||||
scripts/build-check.sh
|
||||
```
|
||||
|
||||
### Multi-Commit Rollback
|
||||
|
||||
If you need to go back several commits:
|
||||
|
||||
```bash
|
||||
# View recent commits
|
||||
git log --oneline -10
|
||||
|
||||
# Rollback to a specific commit (example: abc1234)
|
||||
git reset --hard abc1234
|
||||
|
||||
# Redeploy
|
||||
scripts/deploy.sh
|
||||
```
|
||||
|
||||
### Rollback Verification
|
||||
|
||||
After rolling back, verify the system is stable:
|
||||
|
||||
```bash
|
||||
# Check containers match the previous code
|
||||
scripts/build-check.sh
|
||||
|
||||
# Check API is healthy
|
||||
curl -sf http://localhost:3001/api/health | jq .
|
||||
|
||||
# Check frontend is responsive
|
||||
curl -sf http://localhost:3000/ | head -c 500
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Manual Container Cleanup
|
||||
|
||||
If containers become corrupted or stuck:
|
||||
|
||||
```bash
|
||||
# Stop all containers
|
||||
docker compose down
|
||||
|
||||
# Remove volumes (WARNING: deletes data)
|
||||
docker compose down -v
|
||||
|
||||
# Verify they're gone
|
||||
docker compose ps
|
||||
|
||||
# Full redeploy
|
||||
scripts/deploy.sh
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Monitoring & Logs
|
||||
|
||||
### Deployment Log
|
||||
|
||||
```bash
|
||||
tail -f logs/deploy.log
|
||||
```
|
||||
|
||||
### Container Logs
|
||||
|
||||
```bash
|
||||
# Backend logs
|
||||
docker compose logs gravl-backend
|
||||
|
||||
# Frontend logs
|
||||
docker compose logs gravl-frontend
|
||||
|
||||
# All logs with timestamps
|
||||
docker compose logs --timestamps --follow
|
||||
```
|
||||
|
||||
### Build Info
|
||||
|
||||
```bash
|
||||
# List deployed images
|
||||
docker images | grep gravl
|
||||
|
||||
# Inspect container labels (build metadata)
|
||||
docker inspect gravl-backend | jq '.Config.Labels'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Always test in staging first** — Validate the deploy in a non-production environment
|
||||
2. **Check status before deploying** — Run `scripts/build-check.sh` to ensure no stale containers
|
||||
3. **Review logs after deployment** — Check `logs/deploy.log` for warnings or errors
|
||||
4. **Plan rollbacks** — Know which commits are stable before deploying
|
||||
5. **Monitor health endpoints** — Regularly ping `/api/health` in production
|
||||
6. **Backup before major changes** — Tag releases in git before significant deployments
|
||||
7. **Use semantic commits** — Make it easy to identify which commits introduced changes
|
||||
|
||||
---
|
||||
|
||||
## FAQ
|
||||
|
||||
**Q: Can I deploy without building (e.g., just restart containers)?**
|
||||
A: No. The script always rebuilds to prevent stale code. This is intentional for safety.
|
||||
|
||||
**Q: How long should a deployment take?**
|
||||
A: Typically 60-90 seconds (build time + health check). If longer, check Docker build performance.
|
||||
|
||||
**Q: What if I need to deploy a specific commit?**
|
||||
A: Check it out first, then deploy:
|
||||
```bash
|
||||
git checkout <commit-sha>
|
||||
scripts/deploy.sh
|
||||
```
|
||||
|
||||
**Q: Can I skip the health check?**
|
||||
A: Not recommended. The health check prevents deploying broken code. Fix the health endpoint instead.
|
||||
|
||||
**Q: What data is lost if I rollback?**
|
||||
A: Container rollback only reverts code. Database data persists unless you `docker compose down -v`.
|
||||
|
||||
---
|
||||
|
||||
**Last Updated:** 2026-03-03
|
||||
**Document Version:** 1.0
|
||||
**Phase:** 07-03
|
||||
Reference in New Issue
Block a user