Skip to content

Release Phase

Objective: Ensure a smooth transition from development to production while maintaining high-quality standards and minimizing downtime.

1. Tools

Tool Purpose
Jenkins/GitHub Actions Automate CI/CD pipeline for deployment.
Deployment Ensure consistent environment setup.
Prometheus/Grafana Monitor system health during and after deployment.
Slack/Email Communicate release progress and outcomes with the team.

2. Prerequisites

In order to achieve successful release phase we will need to do the following:

  1. CTO and Team lead team verifies that all critical features pass automated and manual tests.
  2. Ensure all technical and user documentation is up to date.
  3. All core members of the team have approved the release PR.

3. Preparation

  • Feature Verification
  • All critical features pass automated and manual testing.
  • End-to-end tests are executed successfully in staging.
  • Performance and stress testing results meet expectations.

  • Documentation

  • Technical documentation is up to date.
  • User-facing documentation (help guides, release notes) is finalized.
  • API documentation reflects any changes or new endpoints.

  • Approvals

  • Release PR is reviewed and approved by all core team members.
  • Management have been informed and approve the release timeline.

4. Database Migrations

  • Review Migration Scripts
  • Ensure scripts are idempotent and safe for production.
  • Test migrations with a full dataset in the staging environment.

  • Backup

  • Perform a full database backup before applying migrations.
  • Confirm backup integrity and recovery options.

  • Execute Migrations

  • Apply migrations during a low-traffic window.
  • Monitor database performance during and after migration.

  • Validation

  • Confirm schema changes in the database.
  • Perform smoke tests on features dependent on the database.

5. Staging Validation

  • Environment Configuration
  • Staging mirrors production for infrastructure and application versions.
  • Anonymized production data is used for testing.

  • Deployment

  • Deploy the release to the staging environment using the CI/CD pipeline.
  • Verify functionality through end-to-end tests.

  • Logs and Monitoring

  • Analyze staging logs for errors or anomalies.
  • Confirm system metrics (response times, error rates) are within acceptable limits.

6. Deployment

  • Pre-Deployment
  • Announce release timing to the team and management.
  • Ensure rollback scripts and procedures are ready.
  • Confirm database backup is complete and validated.

  • Deployment Steps

  • Create the release branch from master.
  • Trigger the CI/CD pipeline to build and deploy.
  • Apply database migrations.
  • Restart services to load the new codebase and configurations.

  • Validation

  • Conduct post-release smoke tests in production.
  • Verify critical features and endpoints are functional.

7. Post-Release

  • Monitoring
  • Track key metrics in Prometheus/Grafana for 24-48 hours.
  • Set up alerts for critical metrics (e.g., error rates, latency).
  • Analyze logs in Kibana for any unexpected behavior.

  • User Feedback

  • Collect and review feedback from end-users and stakeholders.
  • Document any reported issues for resolution in the next cycle.

  • Team Retrospective

  • Discuss successes and challenges in the release process.
  • Identify improvements for future releases.

  • Documentation Update

  • Update release notes with details about new features, fixes, and changes.
  • Finalize any updates to API or technical documentation.

6. Rollback [Optinal]

  • Execute rollback scripts for the codebase.
  • Restore the database to the pre-migration backup.
  • Communicate rollback details to the team and management.
  • Investigate the cause of the failure and document findings.

7. Checklist

Don't worry regarding release phase, everything will be automated within .sh scripts I've implemented throughout my carreer.

In the end we need to have the following:

  • The release is live, stable, and monitored.
  • All relevant documentation is updated and distributed.
  • Feedback has been collected and analyzed.
  • The team is ready to plan the next iteration.

8. Scripts

These scripts are very approximate for and will vary from project to project. The main scripts which will be updated according to the style and needs of the specific project.

Name Purpose Link
test.sh Run unit tests and integration tests locally. test.sh
backup_db.sh Perform a full database backup before migration or deployment. backup_db.sh
run_migrations.sh Apply database migrations in a controlled environment. run_migrations.sh
generate_docs.sh Generate API documentation automatically from code annotations. generate_docs.sh
validate_dependencies.sh Validate all project dependencies for compatibility. validate_dependencies.sh
rollback.sh Rollback the application and database to the last stable state. rollback.sh
prometheus_alert.sh Configure Prometheus alert rules for monitoring deployment. prometheus_alert.sh
deploy_staging.sh Deploy the application to the staging environment. deploy_staging.sh
analyze_logs.sh Parse logs for errors and send notifications via Slack. analyze_logs.sh
release_report.sh Generate a release report summarizing changes and metrics. release_report.sh

8.1 Script Details

test.sh

#!/bin/bash
# Purpose: Run unit and integration tests
echo "Running tests..."
pytest --junitxml=results.xml
if [ $? -eq 0 ]; then
  echo "All tests passed successfully!"
else
  echo "Some tests failed. Check results.xml for details."
  exit 1
fi

backup_db.sh

#!/bin/bash
# Purpose: Backup the database
TIMESTAMP=$(date +"%F")
BACKUP_DIR="/backups/$TIMESTAMP"
mkdir -p "$BACKUP_DIR"
pg_dump -U $DB_USER -h $DB_HOST $DB_NAME > "$BACKUP_DIR/db_backup.sql"
echo "Database backup completed: $BACKUP_DIR/db_backup.sql"

run_migrations.sh

#!/bin/bash
# Purpose: Apply database migrations
echo "Applying database migrations..."
python manage.py makemigrations
python manage.py migrate
if [ $? -eq 0 ]; then
  echo "Migrations applied successfully!"
else
  echo "Migration failed. Check logs for details."
  exit 1
fi

generate_docs.sh

#!/bin/bash
# Purpose: Generate API documentation
echo "Generating API documentation..."
python manage.py generateschema > openapi.json
if [ $? -eq 0 ]; then
  echo "Documentation generated successfully in openapi.json."
else
  echo "Documentation generation failed."
  exit 1
fi

validate_dependencies.sh

#!/bin/bash
# Purpose: Validate project dependencies
echo "Validating dependencies..."
pip check
if [ $? -eq 0 ]; then
  echo "Dependencies are valid."
else
  echo "Dependency validation failed. Resolve issues above."
  exit 1
fi

rollback.sh

#!/bin/bash
# Purpose: Rollback to the previous stable version
echo "Rolling back release..."
git checkout previous-stable
python manage.py migrate --fake <previous_migration>
if [ $? -eq 0 ]; then
  echo "Rollback completed successfully!"
else
  echo "Rollback failed. Check logs for details."
  exit 1
fi

prometheus_alert.sh

#!/bin/bash
# Purpose: Set up Prometheus alert rules
ALERT_RULES_FILE="/etc/prometheus/alerts.rules"
echo "Setting up Prometheus alert rules..."
cat <<EOT > $ALERT_RULES_FILE
groups:
  - name: release-monitoring
    rules:
      - alert: HighErrorRate
        expr: rate(http_requests_total{status="500"}[5m]) > 0.05
        for: 5m
        labels:
          severity: critical
        annotations:
          summary: "High error rate detected in production."
          description: "Error rate exceeds 5% over the last 5 minutes."
EOT
echo "Prometheus alert rules configured in $ALERT_RULES_FILE."

deploy_staging.sh

#!/bin/bash
# Purpose: Deploy to staging environment
echo "Deploying to staging..."
docker-compose -f docker-compose.staging.yml up -d
if [ $? -eq 0 ]; then
  echo "Deployment to staging completed successfully!"
else
  echo "Deployment to staging failed."
  exit 1
fi

analyze_logs.sh

#!/bin/bash
# Purpose: Analyze logs for errors
LOG_DIR="/var/log/app"
echo "Analyzing logs for errors..."
grep -i "error" $LOG_DIR/*.log | tee error_logs.txt
if [ -s error_logs.txt ]; then
  echo "Errors found. Sending Slack notification..."
  curl -X POST -H 'Content-type: application/json' --data '{"text":"Errors detected in logs. Please investigate!"}' $SLACK_WEBHOOK_URL
else
  echo "No errors found in logs."
fi

release_report.sh

#!/bin/bash
# Purpose: Generate a release report
RELEASE_NOTES_FILE="release_notes.md"
echo "Generating release report..."
cat <<EOT > $RELEASE_NOTES_FILE
# Release Report
## Date: $(date +"%F")

### Changes:
- Feature X implemented
- Bug Y fixed
- Performance improvement Z

### Metrics:
- Deployment duration: X minutes
- Test pass rate: 100%
EOT
echo "Release report saved in $RELEASE_NOTES_FILE."