A 15-Minute Guide to AWS Systems Manager Session Manager (No More SSH)

Key takeaways

Session Manager eliminates SSH keys, bastion hosts, and port 22 security group rules while improving audit trails
Setup requires SSM agent (pre-installed on AWS Linux 2023/Amazon Linux 2) and IAM permissions (5-minute configuration)
CloudWatch Logs integration provides complete session audit trail for compliance (SOC 2, PCI-DSS, HIPAA)
S3 session logging captures full terminal output for forensic analysis and incident response
Port forwarding enables secure RDS access without VPN or bastion host complexity

Your security team flags the audit: 15 engineers have SSH keys with access to production EC2 instances. Key rotation hasn't happened in 18 months. The bastion host running in a public subnet is a single point of failure and costs $50/month. Security group rules allow port 22 from "0.0.0.0/0" with a comment "TODO: restrict this."

This is the SSH anti-pattern: hard to audit, painful to rotate credentials, expensive to maintain bastion infrastructure, and fundamentally insecure. AWS Systems Manager Session Manager solves all these problems with a service that's been available since 2018 but remains underutilized.

This guide provides a step-by-step implementation of Session Manager that you can deploy in 15 minutes, immediately improving your security posture while simplifying operations.

Why Session Manager vs. SSH

Traditional SSH Architecture

┌─────────┐     SSH      ┌─────────┐     SSH      ┌─────────┐
│Engineer │────────────► │ Bastion │────────────► │  EC2    │
│         │              │  Host   │              │ Private │
└─────────┘              └─────────┘              └─────────┘
    ↓                         ↓                        ↓
SSH Key              SSH Key + Audit            SSH Key
Rotation             Security Group             No Audit
No Audit             Maintenance                Overprivileged

Problems:

Three sets of SSH keys to manage and rotate
Bastion host infrastructure costs and maintenance
Security group rules allowing port 22
Limited audit trail (who accessed what, when)
Overprivileged access (SSH gives shell = root access potential)

Session Manager Architecture

┌─────────┐   HTTPS/443  ┌──────────────┐   IAM    ┌─────────┐
│Engineer │─────────────►│Session Manager│─────────►│  EC2    │
│         │              │   Service     │          │ Private │
└─────────┘              └──────────────┘          └─────────┘
    ↓                           ↓                        ↓
IAM Auth              CloudWatch Logs              SSM Agent
MFA                   S3 Logging                   No Keys
Audit Trail           CloudTrail                   VPC Only

Benefits:

Zero SSH keys to manage
No bastion host infrastructure
No port 22 in security groups
Complete audit trail (CloudWatch + S3 + CloudTrail)
IAM-based access control with MFA support
Session recording for compliance
Port forwarding for database access

Setup: The 15-Minute Implementation

Step 1: Verify SSM Agent (2 minutes)

SSM Agent is pre-installed on:

Amazon Linux 2
Amazon Linux 2023
Ubuntu 16.04+
Windows Server 2016+

Check agent status:

# On Amazon Linux 2023
sudo systemctl status amazon-ssm-agent
 
# On Ubuntu
sudo snap list amazon-ssm-agent

If not installed (rare for AWS AMIs):

# Amazon Linux 2023
sudo yum install -y amazon-ssm-agent
sudo systemctl enable amazon-ssm-agent
sudo systemctl start amazon-ssm-agent
 
# Ubuntu
sudo snap install amazon-ssm-agent --classic
sudo snap start amazon-ssm-agent

Step 2: IAM Instance Profile (5 minutes)

EC2 instances need permission to communicate with Session Manager.

Terraform configuration:

# IAM role for EC2 instances
resource "aws_iam_role" "ec2_ssm" {
  name = "ec2-ssm-role"
 
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Action = "sts:AssumeRole"
      Effect = "Allow"
      Principal = {
        Service = "ec2.amazonaws.com"
      }
    }]
  })
}
 
# Attach AWS managed policy for SSM
resource "aws_iam_role_policy_attachment" "ssm_managed_instance" {
  role       = aws_iam_role.ec2_ssm.name
  policy_arn = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
}
 
# Instance profile
resource "aws_iam_instance_profile" "ec2_ssm" {
  name = "ec2-ssm-profile"
  role = aws_iam_role.ec2_ssm.name
}
 
# Attach to EC2 instance
resource "aws_instance" "app_server" {
  ami                    = data.aws_ami.amazon_linux_2023.id
  instance_type          = "t3.micro"
  iam_instance_profile   = aws_iam_instance_profile.ec2_ssm.name
  vpc_security_group_ids = [aws_security_group.app.id]
 
  tags = {
    Name = "app-server"
  }
}
 
# Security group - NO PORT 22!
resource "aws_security_group" "app" {
  name_description = "Application server security group"
  vpc_id           = aws_vpc.main.id
 
  # Only application ports
  ingress {
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = [aws_vpc.main.cidr_block]
  }
 
  # Outbound for SSM communication (HTTPS)
  egress {
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
 
  egress {
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

Step 3: IAM User Permissions (3 minutes)

Engineers need permission to start sessions.

# IAM policy for engineers
resource "aws_iam_policy" "session_manager_users" {
  name        = "SessionManagerUsers"
  description = "Allow engineers to start Session Manager sessions"
 
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = [
          "ssm:StartSession"
        ]
        Resource = [
          "arn:aws:ec2:${var.region}:${var.account_id}:instance/*"
        ]
        Condition = {
          StringLike = {
            "ssm:resourceTag/Environment" = ["staging", "production"]
          }
        }
      },
      {
        Effect = "Allow"
        Action = [
          "ssm:TerminateSession",
          "ssm:ResumeSession"
        ]
        Resource = [
          "arn:aws:ssm:*:*:session/$${aws:username}-*"
        ]
      },
      {
        Effect = "Allow"
        Action = [
          "ssm:DescribeSessions",
          "ssm:GetConnectionStatus"
        ]
        Resource = "*"
      }
    ]
  })
}
 
# Attach to engineers group
resource "aws_iam_group_policy_attachment" "engineers_session_manager" {
  group      = aws_iam_group.engineers.name
  policy_arn = aws_iam_policy.session_manager_users.arn
}

Step 4: Enable Session Logging (5 minutes)

CloudWatch Logs for audit trail:

# CloudWatch log group for session logs
resource "aws_cloudwatch_log_group" "session_manager" {
  name              = "/aws/ssm/session-manager"
  retention_in_days = 90
  kms_key_id        = aws_kms_key.session_logs.arn
}
 
# S3 bucket for session recordings
resource "aws_s3_bucket" "session_logs" {
  bucket = "session-manager-logs-${var.account_id}"
}
 
resource "aws_s3_bucket_versioning" "session_logs" {
  bucket = aws_s3_bucket.session_logs.id
 
  versioning_configuration {
    status = "Enabled"
  }
}
 
resource "aws_s3_bucket_server_side_encryption_configuration" "session_logs" {
  bucket = aws_s3_bucket.session_logs.id
 
  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm     = "aws:kms"
      kms_master_key_id = aws_kms_key.session_logs.arn
    }
  }
}
 
# KMS key for encryption
resource "aws_kms_key" "session_logs" {
  description             = "KMS key for Session Manager logs"
  deletion_window_in_days = 30
  enable_key_rotation     = true
 
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Sid    = "Enable IAM User Permissions"
        Effect = "Allow"
        Principal = {
          AWS = "arn:aws:iam::${var.account_id}:root"
        }
        Action   = "kms:*"
        Resource = "*"
      },
      {
        Sid    = "Allow CloudWatch Logs"
        Effect = "Allow"
        Principal = {
          Service = "logs.${var.region}.amazonaws.com"
        }
        Action = [
          "kms:Encrypt",
          "kms:Decrypt",
          "kms:ReEncrypt*",
          "kms:GenerateDataKey*",
          "kms:CreateGrant",
          "kms:DescribeKey"
        ]
        Resource = "*"
        Condition = {
          ArnLike = {
            "kms:EncryptionContext:aws:logs:arn" = "arn:aws:logs:${var.region}:${var.account_id}:log-group:/aws/ssm/session-manager"
          }
        }
      }
    ]
  })
}
 
# Session Manager preferences
resource "aws_ssm_document" "session_manager_prefs" {
  name            = "SSM-SessionManagerRunShell"
  document_type   = "Session"
  document_format = "JSON"
 
  content = jsonencode({
    schemaVersion = "1.0"
    description   = "Document to hold regional settings for Session Manager"
    sessionType   = "Standard_Stream"
    inputs = {
      s3BucketName                = aws_s3_bucket.session_logs.id
      s3KeyPrefix                 = "session-logs/"
      s3EncryptionEnabled         = true
      cloudWatchLogGroupName      = aws_cloudwatch_log_group.session_manager.name
      cloudWatchEncryptionEnabled = true
      cloudWatchStreamingEnabled  = true
      kmsKeyId                    = aws_kms_key.session_logs.arn
      runAsEnabled                = false
      runAsDefaultUser            = ""
      idleSessionTimeout          = "20"  # 20 minutes
      maxSessionDuration          = ""    # Unlimited
      shellProfile = {
        linux = "cd /home/ec2-user && exec bash"
      }
    }
  })
}

Using Session Manager

Method 1: AWS Console (Easiest)

Navigate to EC2 Console
Select instance
Click "Connect" → "Session Manager" → "Connect"
Terminal opens in browser

Method 2: AWS CLI (Recommended)

# Install session-manager-plugin first
# macOS
brew install --cask session-manager-plugin
 
# Linux
curl "https://s3.amazonaws.com/session-manager-downloads/plugin/latest/linux_64bit/session-manager-plugin.rpm" -o "session-manager-plugin.rpm"
sudo yum install -y session-manager-plugin.rpm
 
# Start session
aws ssm start-session --target i-1234567890abcdef0
 
# Start session as specific user (requires runAsEnabled=true)
aws ssm start-session \
  --target i-1234567890abcdef0 \
  --document-name AWS-StartInteractiveCommand \
  --parameters command="sudo su - ubuntu"

Method 3: SSH-like Wrapper

Create an SSH config wrapper for familiar syntax:

# ~/.ssh/config
Host i-* mi-*
    ProxyCommand sh -c "aws ssm start-session --target %h --document-name AWS-StartSSHSession --parameters 'portNumber=%p'"
    User ec2-user
 
# Now use familiar SSH syntax
ssh i-1234567890abcdef0

Advanced Patterns

Pattern 1: Port Forwarding to Private RDS

Access RDS databases without VPN or bastion host:

# Forward local port 5432 to RDS endpoint
aws ssm start-session \
  --target i-1234567890abcdef0 \
  --document-name AWS-StartPortForwardingSessionToRemoteHost \
  --parameters '{
    "portNumber":["5432"],
    "localPortNumber":["5432"],
    "host":["mydb.cluster-abc123.us-east-1.rds.amazonaws.com"]
  }'
 
# In another terminal, connect to RDS via localhost
psql -h localhost -p 5432 -U myuser -d production

Use cases:

Database administration without VPN
Development/debugging access to RDS
Secure tunneling to ElastiCache, OpenSearch, etc.

Pattern 2: Session Manager for ECS Tasks

Connect to running ECS Fargate containers:

# Enable ECS Exec on service
resource "aws_ecs_service" "app" {
  name            = "app-service"
  cluster         = aws_ecs_cluster.main.id
  task_definition = aws_ecs_task_definition.app.arn
 
  enable_execute_command = true  # Enable Session Manager
 
  # Task role needs SSM permissions
  task_definition = aws_ecs_task_definition.app.arn
}
 
# Task role with SSM permissions
resource "aws_iam_role" "ecs_task" {
  name = "ecs-task-role"
 
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Action = "sts:AssumeRole"
      Effect = "Allow"
      Principal = {
        Service = "ecs-tasks.amazonaws.com"
      }
    }]
  })
}
 
resource "aws_iam_role_policy_attachment" "ecs_task_ssm" {
  role       = aws_iam_role.ecs_task.name
  policy_arn = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
}

Access ECS container:

# List running tasks
aws ecs list-tasks --cluster production --service-name app-service
 
# Start session to container
aws ecs execute-command \
  --cluster production \
  --task arn:aws:ecs:us-east-1:123456789012:task/production/abc123 \
  --container app \
  --interactive \
  --command "/bin/bash"

Pattern 3: Run Commands Without Interactive Session

Execute commands across multiple instances:

# Run command on multiple instances
aws ssm send-command \
  --document-name "AWS-RunShellScript" \
  --targets "Key=tag:Environment,Values=production" \
  --parameters 'commands=["df -h", "free -m", "uptime"]' \
  --output-s3-bucket-name "session-manager-logs-123456789012" \
  --output-s3-key-prefix "run-command"
 
# Check command status
aws ssm list-command-invocations \
  --command-id abc-123-xyz \
  --details

Pattern 4: Emergency Break-Glass Access

Configure emergency access with MFA requirement:

# IAM policy requiring MFA for Session Manager
resource "aws_iam_policy" "session_manager_mfa_required" {
  name        = "SessionManagerMFARequired"
  description = "Require MFA for Session Manager access"
 
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Sid    = "AllowSessionManagerWithMFA"
        Effect = "Allow"
        Action = [
          "ssm:StartSession"
        ]
        Resource = "*"
        Condition = {
          BoolIfExists = {
            "aws:MultiFactorAuthPresent" = "true"
          }
          NumericLessThan = {
            "aws:MultiFactorAuthAge" = "3600"  # MFA valid for 1 hour
          }
        }
      },
      {
        Sid    = "DenySessionManagerWithoutMFA"
        Effect = "Deny"
        Action = [
          "ssm:StartSession"
        ]
        Resource = "*"
        Condition = {
          BoolIfExists = {
            "aws:MultiFactorAuthPresent" = "false"
          }
        }
      }
    ]
  })
}

Monitoring and Alerts

CloudWatch Insights Queries

-- All sessions started in last 24 hours
fields @timestamp, userIdentity.principalId, requestParameters.target
| filter eventName = "StartSession"
| sort @timestamp desc
| limit 100
 
-- Sessions by user
fields @timestamp, userIdentity.principalId, count() as SessionCount
| filter eventName = "StartSession"
| stats count() by userIdentity.principalId
| sort SessionCount desc
 
-- Long-running sessions (>1 hour)
fields @timestamp, userIdentity.principalId, requestParameters.target, sessionDuration
| filter eventName = "TerminateSession" and sessionDuration > 3600
| sort @timestamp desc

Alerting on Suspicious Activity

# CloudWatch metric filter for session starts
resource "aws_cloudwatch_log_metric_filter" "session_starts" {
  name           = "session-manager-starts"
  log_group_name = "/aws/ssm/session-manager"
  pattern        = "[eventName = StartSession]"
 
  metric_transformation {
    name      = "SessionStarts"
    namespace = "SessionManager"
    value     = "1"
  }
}
 
# Alert on unusual session activity
resource "aws_cloudwatch_metric_alarm" "unusual_sessions" {
  alarm_name          = "unusual-session-activity"
  comparison_operator = "GreaterThanThreshold"
  evaluation_periods  = 1
  metric_name         = "SessionStarts"
  namespace           = "SessionManager"
  period              = 300
  statistic           = "Sum"
  threshold           = 10  # More than 10 sessions in 5 minutes
  alarm_description   = "Unusual Session Manager activity detected"
  alarm_actions       = [aws_sns_topic.security_alerts.arn]
}
 
# Alert on after-hours access
resource "aws_cloudwatch_metric_alarm" "after_hours_sessions" {
  alarm_name          = "after-hours-session-access"
  comparison_operator = "GreaterThanThreshold"
  evaluation_periods  = 1
  metric_name         = "SessionStarts"
  namespace           = "SessionManager"
  period              = 300
  statistic           = "Sum"
  threshold           = 0
  alarm_description   = "Session started outside business hours"
  alarm_actions       = [aws_sns_topic.security_alerts.arn]
 
  # Only alert between 6 PM - 8 AM weekdays
  treat_missing_data = "notBreaching"
}

Compliance and Audit

SOC 2 / PCI-DSS Requirements

Session Manager satisfies key control requirements:

Access Control (AC):

✅ IAM-based authentication
✅ MFA enforcement capability
✅ Least-privilege access via IAM policies
✅ Temporary session credentials (no persistent keys)

Audit and Accountability (AU):

✅ CloudTrail logs every session start/terminate
✅ CloudWatch Logs capture session activity
✅ S3 session recordings for forensics
✅ User attribution (who accessed what, when)

Configuration Management (CM):

✅ No SSH keys to manage
✅ No bastion host patches
✅ Automated compliance via IAM policies

Compliance Reports

Generate monthly access reports:

# Generate session access report
aws cloudtrail lookup-events \
  --lookup-attributes AttributeKey=EventName,AttributeValue=StartSession \
  --start-time $(date -u -d '30 days ago' +%Y-%m-%dT%H:%M:%S) \
  --end-time $(date -u +%Y-%m-%dT%H:%M:%S) \
  --max-results 1000 \
  --query 'Events[].{Time:EventTime,User:Username,Instance:Resources[0].ResourceName}' \
  --output table
 
# Export to CSV for audit
aws cloudtrail lookup-events \
  --lookup-attributes AttributeKey=EventName,AttributeValue=StartSession \
  --start-time $(date -u -d '30 days ago' +%Y-%m-%dT%H:%M:%S) \
  --output json | \
  jq -r '.Events[] | [.EventTime, .Username, (.Resources[0].ResourceName // "N/A")] | @csv' > sessions.csv

Troubleshooting

Issue 1: "TargetNotConnected"

Cause: SSM Agent not running or no IAM instance profile

Solution:

# Check agent status on instance (if you still have SSH)
sudo systemctl status amazon-ssm-agent
 
# Check CloudWatch logs
aws logs tail /aws/ssm/errors --follow
 
# Verify instance profile attached
aws ec2 describe-instances \
  --instance-ids i-1234567890abcdef0 \
  --query 'Reservations[0].Instances[0].IamInstanceProfile'

Issue 2: Session immediately terminates

Cause: IAM permissions insufficient or session preferences misconfigured

Solution:

# Test IAM permissions
aws ssm start-session --target i-1234567890abcdef0 --dry-run
 
# Check session preferences
aws ssm get-document --name SSM-SessionManagerRunShell

Issue 3: Can't access private resources via port forwarding

Cause: Security group or network ACL blocking traffic

Solution:

# Verify security group allows outbound HTTPS
aws ec2 describe-security-groups \
  --group-ids sg-abc123 \
  --query 'SecurityGroups[0].IpPermissionsEgress'
 
# Test connectivity from EC2 instance
aws ssm start-session --target i-1234567890abcdef0
# Inside session:
curl -I https://www.google.com
telnet mydb.cluster-abc123.us-east-1.rds.amazonaws.com 5432

Migration Path: SSH → Session Manager

Week 1: Pilot (Non-Production)

Deploy Session Manager to dev/staging
Train team on AWS CLI session-manager-plugin
Test port forwarding for RDS access

Week 2: Production Deployment

Attach IAM instance profiles to production EC2
Configure session logging (CloudWatch + S3)
Update runbooks with Session Manager commands

Week 3: Dual-Mode Operation

Keep SSH access enabled alongside Session Manager
Monitor adoption and identify issues
Gather team feedback

Week 4: SSH Deprecation

Remove port 22 from security groups
Decommission bastion hosts
Revoke SSH keys
Celebrate improved security posture 🎉

Cost Savings

Before Session Manager:

Bastion host (t3.small): $15/month
Bastion host data transfer: $10/month
EIP for bastion: $3.60/month
Engineer time managing keys: 2 hours/month @ $100/hour = $200
Total: $228.60/month

After Session Manager:

Session Manager API calls: $0 (included in SSM)
S3 storage for logs: ~$1/month
CloudWatch Logs: ~$2/month
Total: $3/month

Annual savings: $2,707

Conclusion: Eliminate SSH Keys Today

Session Manager isn't just a security improvement—it's operational simplification. No more:

SSH key rotation
Bastion host patching
Security group port 22 exceptions
VPN client configuration
Access request workflows

The 15-minute setup delivers immediate value:

Improved security (no keys, complete audit trail)
Reduced cost (no bastion infrastructure)
Better compliance (SOC 2, PCI-DSS controls)
Simplified operations (IAM-based access)

If you're still using SSH for EC2 access in 2025, you're paying a security and operational tax that Session Manager eliminates for free.

Action Items:

Audit current EC2 instances using SSH (port 22 in security groups)
Enable Session Manager on one non-production instance (verify SSM agent, attach IAM profile)
Configure session logging to CloudWatch and S3
Test port forwarding to RDS for database access
Update team runbooks with Session Manager commands
Create IAM policies restricting Session Manager by environment tag
Schedule SSH deprecation for 30 days from now
Decommission bastion hosts and celebrate cost savings

Key takeaways

Why Session Manager vs. SSH

Traditional SSH Architecture

Session Manager Architecture

Setup: The 15-Minute Implementation

Step 1: Verify SSM Agent (2 minutes)

Step 2: IAM Instance Profile (5 minutes)

Step 3: IAM User Permissions (3 minutes)

Step 4: Enable Session Logging (5 minutes)

Using Session Manager

Method 1: AWS Console (Easiest)

Method 2: AWS CLI (Recommended)

Method 3: SSH-like Wrapper

Advanced Patterns

Pattern 1: Port Forwarding to Private RDS

Pattern 2: Session Manager for ECS Tasks

Pattern 3: Run Commands Without Interactive Session

Pattern 4: Emergency Break-Glass Access

Monitoring and Alerts

CloudWatch Insights Queries

Alerting on Suspicious Activity

Compliance and Audit

SOC 2 / PCI-DSS Requirements

Compliance Reports

Troubleshooting

Issue 1: "TargetNotConnected"

Issue 2: Session immediately terminates

Issue 3: Can't access private resources via port forwarding

Migration Path: SSH → Session Manager

Cost Savings

Conclusion: Eliminate SSH Keys Today

Need Help with Your Cloud Infrastructure?