Deployment Process & CI/CD

Overview

This document outlines the deployment process, CI/CD pipelines, and infrastructure management for our SaaS CRM application.

Deployment Architecture

Environment Structure

Production (PROD)
├── Load Balancer (AWS ALB)
├── Web Servers (EC2 Auto-scaling)
│   ├── Nuxt Application (3 instances min)
│   └── Nginx Reverse Proxy
├── API Servers (EC2 Auto-scaling)
│   ├── PHP Phalcon API (3 instances min)
│   └── PHP-FPM
├── Database Cluster
│   ├── MySQL Primary (RDS Multi-AZ)
│   └── MySQL Read Replicas (2)
├── Cache Layer
│   ├── Redis Cluster (ElastiCache)
│   └── CDN (CloudFront)
└── Supporting Services
    ├── RabbitMQ (Amazon MQ)
    ├── Elasticsearch (AWS OpenSearch)
    └── S3 (File Storage)

Staging (STG)
├── Similar to production (smaller scale)
└── Single instances for cost optimization

Development (DEV)
├── Single server setup
└── Docker Compose environment

CI/CD Pipeline

GitHub Actions Workflow

# .github/workflows/deploy.yml
name: Deploy Pipeline

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

env:
  NODE_VERSION: '18'
  PHP_VERSION: '8.2'

jobs:
  # 1. Code Quality Checks
  quality:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Setup Node.js
        uses: actions/setup-node@v3
        with:
          node-version: ${{ env.NODE_VERSION }}

      - name: Setup PHP
        uses: shivammathur/setup-php@v2
        with:
          php-version: ${{ env.PHP_VERSION }}
          extensions: phalcon, mbstring, mysql

      - name: Install dependencies
        run: |
          npm ci
          composer install --no-dev

      - name: Run linting
        run: |
          npm run lint
          composer run-script lint

      - name: Run security audit
        run: |
          npm audit
          composer audit

  # 2. Testing
  test:
    runs-on: ubuntu-latest
    needs: quality

    services:
      mysql:
        image: mysql:8.0
        env:
          MYSQL_ROOT_PASSWORD: test
          MYSQL_DATABASE: crm_test
        options: >-
          --health-cmd="mysqladmin ping"
          --health-interval=10s
          --health-timeout=5s
          --health-retries=5

      redis:
        image: redis:7
        options: >-
          --health-cmd="redis-cli ping"
          --health-interval=10s
          --health-timeout=5s
          --health-retries=5

    steps:
      - uses: actions/checkout@v3

      - name: Setup test environment
        run: |
          cp .env.test .env
          npm ci
          composer install

      - name: Run database migrations
        run: |
          php artisan migrate --env=test

      - name: Run unit tests
        run: |
          npm run test:unit
          ./vendor/bin/phpunit --testsuite=unit

      - name: Run integration tests
        run: |
          npm run test:integration
          ./vendor/bin/phpunit --testsuite=integration

      - name: Upload coverage
        uses: codecov/codecov-action@v3
        with:
          files: ./coverage/lcov.info,./coverage.xml

  # 3. Build
  build:
    runs-on: ubuntu-latest
    needs: test
    if: github.ref == 'refs/heads/main' || github.ref == 'refs/heads/develop'

    steps:
      - uses: actions/checkout@v3

      - name: Setup build environment
        run: |
          npm ci
          composer install --no-dev --optimize-autoloader

      - name: Build frontend
        run: |
          npm run build

      - name: Build backend
        run: |
          composer dump-autoload --optimize
          php artisan config:cache
          php artisan route:cache

      - name: Build Docker images
        run: |
          docker build -t crm-frontend:${{ github.sha }} -f docker/frontend/Dockerfile .
          docker build -t crm-backend:${{ github.sha }} -f docker/backend/Dockerfile .

      - name: Push to ECR
        env:
          AWS_REGION: us-east-1
        run: |
          aws ecr get-login-password | docker login --username AWS --password-stdin $ECR_REGISTRY
          docker tag crm-frontend:${{ github.sha }} $ECR_REGISTRY/crm-frontend:${{ github.sha }}
          docker tag crm-backend:${{ github.sha }} $ECR_REGISTRY/crm-backend:${{ github.sha }}
          docker push $ECR_REGISTRY/crm-frontend:${{ github.sha }}
          docker push $ECR_REGISTRY/crm-backend:${{ github.sha }}

  # 4. Deploy to Staging
  deploy-staging:
    runs-on: ubuntu-latest
    needs: build
    if: github.ref == 'refs/heads/develop'
    environment:
      name: staging
      url: https://staging.ourcrm.com

    steps:
      - name: Deploy to staging
        run: |
          aws ecs update-service \
            --cluster staging-cluster \
            --service crm-frontend \
            --force-new-deployment

          aws ecs update-service \
            --cluster staging-cluster \
            --service crm-backend \
            --force-new-deployment

      - name: Run smoke tests
        run: |
          npm run test:smoke -- --url=https://staging.ourcrm.com

      - name: Notify team
        uses: 8398a7/action-slack@v3
        with:
          status: ${{ job.status }}
          text: 'Staging deployment completed'
          webhook_url: ${{ secrets.SLACK_WEBHOOK }}

  # 5. Deploy to Production
  deploy-production:
    runs-on: ubuntu-latest
    needs: build
    if: github.ref == 'refs/heads/main'
    environment:
      name: production
      url: https://ourcrm.com

    steps:
      - name: Create deployment
        uses: chrnorm/deployment-action@v2
        with:
          token: ${{ secrets.GITHUB_TOKEN }}
          environment: production

      - name: Blue-Green Deployment
        run: |
          # Deploy to green environment
          ./scripts/deploy-green.sh ${{ github.sha }}

          # Run health checks
          ./scripts/health-check.sh green

          # Switch traffic
          ./scripts/switch-traffic.sh green

          # Monitor for 5 minutes
          ./scripts/monitor.sh 300

      - name: Rollback on failure
        if: failure()
        run: |
          ./scripts/switch-traffic.sh blue
          ./scripts/notify-rollback.sh

Deployment Scripts

Blue-Green Deployment

#!/bin/bash
# scripts/deploy-green.sh

VERSION=$1
ENVIRONMENT="green"

echo "Deploying version $VERSION to $ENVIRONMENT environment"

# Update task definition
aws ecs register-task-definition \
  --family crm-app-$ENVIRONMENT \
  --container-definitions "[{
    \"name\": \"frontend\",
    \"image\": \"$ECR_REGISTRY/crm-frontend:$VERSION\",
    \"memory\": 512,
    \"cpu\": 256,
    \"essential\": true
  },{
    \"name\": \"backend\",
    \"image\": \"$ECR_REGISTRY/crm-backend:$VERSION\",
    \"memory\": 1024,
    \"cpu\": 512,
    \"essential\": true
  }]"

# Update service
aws ecs update-service \
  --cluster production-cluster \
  --service crm-$ENVIRONMENT \
  --task-definition crm-app-$ENVIRONMENT

# Wait for deployment
aws ecs wait services-stable \
  --cluster production-cluster \
  --services crm-$ENVIRONMENT

echo "Deployment complete"

Health Check Script

#!/bin/bash
# scripts/health-check.sh

ENVIRONMENT=$1
URL="https://$ENVIRONMENT.internal.ourcrm.com"

echo "Running health checks for $ENVIRONMENT"

# API health check
API_STATUS=$(curl -s -o /dev/null -w "%{http_code}" $URL/api/health)
if [ $API_STATUS -ne 200 ]; then
  echo "API health check failed: $API_STATUS"
  exit 1
fi

# Database connectivity
DB_CHECK=$(curl -s $URL/api/health/db | jq -r .status)
if [ "$DB_CHECK" != "healthy" ]; then
  echo "Database check failed"
  exit 1
fi

# Redis connectivity
REDIS_CHECK=$(curl -s $URL/api/health/redis | jq -r .status)
if [ "$REDIS_CHECK" != "healthy" ]; then
  echo "Redis check failed"
  exit 1
fi

echo "All health checks passed"

Traffic Switch Script

#!/bin/bash
# scripts/switch-traffic.sh

TARGET=$1  # blue or green

echo "Switching traffic to $TARGET environment"

# Update ALB target group
aws elbv2 modify-rule \
  --rule-arn $PROD_ALB_RULE_ARN \
  --actions Type=forward,TargetGroupArn=$TARGET_GROUP_ARN

# Update Route53 weighted routing
aws route53 change-resource-record-sets \
  --hosted-zone-id $HOSTED_ZONE_ID \
  --change-batch "{
    \"Changes\": [{
      \"Action\": \"UPSERT\",
      \"ResourceRecordSet\": {
        \"Name\": \"api.ourcrm.com\",
        \"Type\": \"A\",
        \"SetIdentifier\": \"$TARGET\",
        \"Weight\": 100,
        \"AliasTarget\": {
          \"HostedZoneId\": \"$ALB_ZONE_ID\",
          \"DNSName\": \"$TARGET-alb.ourcrm.com\",
          \"EvaluateTargetHealth\": true
        }
      }
    }]
  }"

echo "Traffic switched to $TARGET"

Docker Configuration

Frontend Dockerfile

# docker/frontend/Dockerfile
FROM node:18-alpine AS builder

WORKDIR /app

# Copy package files
COPY package*.json ./
RUN npm ci --only=production

# Copy application files
COPY . .

# Build application
RUN npm run build

# Production image
FROM node:18-alpine

WORKDIR /app

# Copy built application
COPY --from=builder /app/.output .output
COPY --from=builder /app/node_modules node_modules

# Health check
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
  CMD node healthcheck.js

EXPOSE 3000

CMD ["node", ".output/server/index.mjs"]

Backend Dockerfile

# docker/backend/Dockerfile
FROM php:8.2-fpm-alpine

# Install extensions
RUN docker-php-ext-install pdo_mysql opcache
RUN pecl install phalcon redis && \
    docker-php-ext-enable phalcon redis

# Install composer
COPY --from=composer:latest /usr/bin/composer /usr/bin/composer

WORKDIR /var/www/html

# Copy application
COPY . .

# Install dependencies
RUN composer install --no-dev --optimize-autoloader

# Configure PHP
COPY docker/backend/php.ini /usr/local/etc/php/conf.d/custom.ini

# Health check
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
  CMD curl -f http://localhost/health || exit 1

EXPOSE 9000

CMD ["php-fpm"]

Docker Compose

# docker-compose.yml
version: '3.8'

services:
  frontend:
    build:
      context: .
      dockerfile: docker/frontend/Dockerfile
    ports:
      - "3000:3000"
    environment:
      NUXT_PUBLIC_API_BASE: http://backend:9000
    depends_on:
      - backend
    networks:
      - crm-network

  backend:
    build:
      context: .
      dockerfile: docker/backend/Dockerfile
    ports:
      - "9000:9000"
    environment:
      DB_HOST: mysql
      DB_DATABASE: crm
      DB_USERNAME: root
      DB_PASSWORD: secret
      REDIS_HOST: redis
    depends_on:
      - mysql
      - redis
    networks:
      - crm-network

  nginx:
    image: nginx:alpine
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./docker/nginx/nginx.conf:/etc/nginx/nginx.conf
      - ./docker/nginx/ssl:/etc/nginx/ssl
    depends_on:
      - frontend
      - backend
    networks:
      - crm-network

  mysql:
    image: mysql:8.0
    environment:
      MYSQL_ROOT_PASSWORD: secret
      MYSQL_DATABASE: crm
    volumes:
      - mysql-data:/var/lib/mysql
    ports:
      - "3306:3306"
    networks:
      - crm-network

  redis:
    image: redis:7-alpine
    ports:
      - "6379:6379"
    volumes:
      - redis-data:/data
    networks:
      - crm-network

volumes:
  mysql-data:
  redis-data:

networks:
  crm-network:
    driver: bridge

Kubernetes Deployment

Deployment Manifest

# k8s/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: crm-frontend
  namespace: production
spec:
  replicas: 3
  selector:
    matchLabels:
      app: crm-frontend
  template:
    metadata:
      labels:
        app: crm-frontend
    spec:
      containers:
      - name: frontend
        image: crm-frontend:latest
        ports:
        - containerPort: 3000
        env:
        - name: NUXT_PUBLIC_API_BASE
          valueFrom:
            configMapKeyRef:
              name: crm-config
              key: api.base.url
        resources:
          requests:
            memory: "256Mi"
            cpu: "250m"
          limits:
            memory: "512Mi"
            cpu: "500m"
        livenessProbe:
          httpGet:
            path: /health
            port: 3000
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /ready
            port: 3000
          initialDelaySeconds: 5
          periodSeconds: 5
---
apiVersion: v1
kind: Service
metadata:
  name: crm-frontend-service
  namespace: production
spec:
  selector:
    app: crm-frontend
  ports:
  - protocol: TCP
    port: 80
    targetPort: 3000
  type: LoadBalancer

Horizontal Pod Autoscaler

# k8s/hpa.yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: crm-frontend-hpa
  namespace: production
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: crm-frontend
  minReplicas: 3
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 80

Infrastructure as Code

Terraform Configuration

# terraform/main.tf
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }

  backend "s3" {
    bucket = "crm-terraform-state"
    key    = "production/terraform.tfstate"
    region = "us-east-1"
  }
}

# VPC Configuration
module "vpc" {
  source = "terraform-aws-modules/vpc/aws"

  name = "crm-vpc"
  cidr = "10.0.0.0/16"

  azs             = ["us-east-1a", "us-east-1b", "us-east-1c"]
  private_subnets = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
  public_subnets  = ["10.0.101.0/24", "10.0.102.0/24", "10.0.103.0/24"]

  enable_nat_gateway = true
  enable_vpn_gateway = true

  tags = {
    Environment = "production"
    Application = "CRM"
  }
}

# RDS MySQL
resource "aws_db_instance" "mysql" {
  identifier     = "crm-mysql"
  engine         = "mysql"
  engine_version = "8.0"
  instance_class = "db.r5.xlarge"

  allocated_storage     = 100
  storage_encrypted     = true
  storage_type          = "gp3"

  db_name  = "crm"
  username = var.db_username
  password = var.db_password

  vpc_security_group_ids = [aws_security_group.rds.id]
  db_subnet_group_name   = aws_db_subnet_group.main.name

  backup_retention_period = 30
  backup_window          = "03:00-04:00"
  maintenance_window     = "sun:04:00-sun:05:00"

  multi_az               = true
  publicly_accessible    = false

  enabled_cloudwatch_logs_exports = ["error", "general", "slowquery"]

  tags = {
    Name        = "CRM Database"
    Environment = "production"
  }
}

# ElastiCache Redis
resource "aws_elasticache_cluster" "redis" {
  cluster_id           = "crm-redis"
  engine              = "redis"
  node_type           = "cache.r6g.xlarge"
  num_cache_nodes     = 1
  parameter_group_name = "default.redis7"
  port                = 6379

  subnet_group_name = aws_elasticache_subnet_group.main.name
  security_group_ids = [aws_security_group.redis.id]

  snapshot_retention_limit = 5
  snapshot_window         = "03:00-05:00"

  tags = {
    Name        = "CRM Cache"
    Environment = "production"
  }
}

# Auto Scaling Group
resource "aws_autoscaling_group" "app" {
  name                = "crm-app-asg"
  vpc_zone_identifier = module.vpc.private_subnets
  target_group_arns   = [aws_lb_target_group.app.arn]
  health_check_type   = "ELB"
  min_size            = 3
  max_size            = 10
  desired_capacity    = 3

  launch_template {
    id      = aws_launch_template.app.id
    version = "$Latest"
  }

  tag {
    key                 = "Name"
    value               = "CRM App Server"
    propagate_at_launch = true
  }
}

# Application Load Balancer
resource "aws_lb" "main" {
  name               = "crm-alb"
  internal           = false
  load_balancer_type = "application"
  security_groups    = [aws_security_group.alb.id]
  subnets           = module.vpc.public_subnets

  enable_deletion_protection = true
  enable_http2              = true

  tags = {
    Name        = "CRM Load Balancer"
    Environment = "production"
  }
}

Deployment Checklist

Pre-Deployment

• Code review completed
• All tests passing
• Security scan completed
• Performance testing done
• Documentation updated
• Database migrations prepared
• Rollback plan documented
• Team notified

During Deployment

• Maintenance page enabled (if needed)
• Database backup taken
• Deployment started
• Health checks passing
• Smoke tests completed
• Monitoring active

Post-Deployment

• Application fully functional
• Performance metrics normal
• Error rates acceptable
• Customer communication sent
• Deployment documented
• Lessons learned captured

Rollback Procedures

Immediate Rollback

#!/bin/bash
# Immediate rollback to previous version

PREVIOUS_VERSION=$(aws ecs describe-services \
  --cluster production-cluster \
  --services crm-app \
  --query 'services[0].taskDefinition' \
  --output text | sed 's/.*://')

echo "Rolling back to version $PREVIOUS_VERSION"

aws ecs update-service \
  --cluster production-cluster \
  --service crm-app \
  --task-definition crm-app:$((PREVIOUS_VERSION - 1))

# Wait for rollback
aws ecs wait services-stable \
  --cluster production-cluster \
  --services crm-app

echo "Rollback complete"

Database Rollback

-- Rollback migration
START TRANSACTION;

-- Reverse schema changes
ALTER TABLE customers DROP COLUMN new_field;

-- Restore data if needed
UPDATE customers SET status = 'active' 
WHERE status = 'migrated';

-- Verify rollback
SELECT COUNT(*) FROM customers WHERE status = 'migrated';

COMMIT;

Monitoring & Alerts

CloudWatch Alarms

# cloudwatch/alarms.yaml
HighErrorRate:
  MetricName: 4XXError
  Threshold: 10
  Period: 300
  EvaluationPeriods: 2
  Actions:
    - !Ref SNSAlertTopic

HighLatency:
  MetricName: TargetResponseTime
  Threshold: 1000  # milliseconds
  Period: 300
  EvaluationPeriods: 2
  Actions:
    - !Ref SNSAlertTopic

LowDiskSpace:
  MetricName: DiskSpaceUtilization
  Threshold: 80  # percentage
  Period: 300
  EvaluationPeriods: 1
  Actions:
    - !Ref SNSAlertTopic

---

Last Updated: January 2025 Version: 1.0.0