Ops at Scale Maturity assesment¶

Estimated time to read: 12 minutes

Based on the provided information, here's a summary of the different levels of maturity across various aspects of software delivery

Culture - Team Structure & Product Alignment¶

Baseline Uncommunicated E2E vision, low collaboration, sub-innovating, multiple teams in silos, reactive.
Novice Clear delivery requirements, rapid intra-team communication, innovation by necessity, teams organized around deliverables, written knowledge.
Intermediate Autonomous teams, articulated business goals, clear project requirements, rapid inter-team communication, innovation by design, maintaining scripts and creating orchestrations, automation as documentation.
Advanced Articulated business vision, clear product requirements, frequent collaborative communication, strategic innovation, shared responsibilities within the autonomous team, shared knowledge.
Expert Articulated business strategy, clear business requirements, rapid and continuous feedback, ownership mindset, proactive, common knowledge transfer.

Deployment Automation¶

Baseline Manual deployments, fully scripted deployments.
Novice Push-button deployments to test environments, manual deployments to production.
Intermediate Immutable structure defined, standard deployments across environments, push-button deployments to production.
Advanced Automated deployments to production after tests pass, zero-downtime deployments, database deployments, independent multi-tier deployments.
Expert Ability to implement continuous deployment.

Continuous Integration¶

Baseline Official builds are performed on developers' machines, build artifacts are managed, some manual scripts still used.
Novice Automated unit tests embedded in the CI phase, coverage is measured and breaks the CI phase, system polls source control and builds on commit, official builds are not performed on developers' machines, build artifacts are automatically managed, dependencies are managed in a repository, build status notification is sent to committer, build only once.
Intermediate Mocked smoke, contract and functional test as part of the CI, SAST is part of the CI, source control tells system when to build, no polling.
Advanced Mocked DAST as part of the CI, code analysis quality gates break the CI phase, SAST and DAST quality gates break the CI phase, container builds become the main CI outcome, trend reports are automatically generated from build server events, fast feedback to developers.
Expert Gated commits to mainline based on CI phase result.

Loosely Coupled Architecture¶

Baseline Monolithic application, risky deployments as all software is deployed/released together.
Novice Monolithic application is broken into smaller modules/components, all product tests must be run on each commit, scaling application requires a complete copy of all components.
Intermediate Pyramid of testing, independent release of components, teams become autonomous, components are containerized (e.g., in Docker), devs/testers need only basic docker commands to run the entire product.
Advanced Different versions of individual components can be run in parallel, support for canary and blue/green release by component, scaling of application at microservice level and more efficient use of resources, inter-service tests use mocking and services can be tested in isolation and in parallel, commit to an individual microservice only runs related tests, faster MTTR.
Expert Loosely coupled microservices, each component can be deployed and released independently, low-risk deployments/releases, contract testing ensures that if tests are passed, the release of microservices is safe.

Version Control & Trunk-based Deployment¶

Baseline No single source of truth, long-lived branches, and disconnected sources of truth.
Intermediate Branches are used for isolating work, and trunk-based branching methods are employed.
Advanced Short-lived branches, single source of truth, and mainline branch is pristine.
Expert All commits are tied to tasks/stories, and pre-release tagging and traceability are in place.
Cutting-edge Product and multi-component interconnected sources of truth, and traceability analysis with auto-generated release notes.

Continuous Testing¶

Baseline Automatic unit testing with every build, and manual functional testing.
Intermediate Code coverage is measured, and peer-reviews are conducted.
Advanced Automated smoke tests, functional and integration testing, security static code analysis (SAST), and automated end-to-end/regression tests in target environments.
Expert Automated performance and security tests (DAST/AST) in target environments, integrated management and maintenance of test data, and shift-right paradigm with test monitoring, alerting, and logging tools.
Cutting-edge Production testing, post-release A/B testing, chaos testing, and business impact testing.

Test Data Management¶

Baseline Shared test data environments and manual data anonymization.
Novice Isolated test data environments and automated data anonymization.
Intermediate Test data management strategy in place, synthetic data generation, and data anonymization tools.
Advanced Test data management integrated into the pipeline, automated synthetic data generation, and consistent data anonymization.
Expert Test data strategy as code, on-demand test data provisioning, and data anonymization integrated into the pipeline.

Monitoring & Observability¶

Baseline Whitebox monitoring performed, basic logs and metrics are recorded locally.
Intermediate Detailed metrics and log data are recorded centrally.
Advanced Blackbox monitoring, access to logs and metric data for teams, and automatic alerts and alarms triggered by metric thresholds.
Expert Constant deployment feedback, profiling against data, predicting potential issues, and debugging implicit failure modes.
Cutting-edge Patterns, traces, and anomalies detected automatically, proactive actions, and dependency analysis of services.

Database Automation¶

Baseline Manual data/schema migrations and manual rollback-enabled data/schema migrations.
Intermediate Automated un-versioned data/schema migrations.
Advanced Database schema changes are version controlled, and DBA team does peer review.
Expert Automated database deployments, data migration in production environments, test data management, and automated rollback-enabled data/schema migrations.
Cutting-edge Zero downtime database releases using canary.

Security¶

Baseline Manual penetration testing.
Intermediate Account & access key compromise detection, compliance scanning, storage monitoring, and configuration monitoring.
Advanced Code vulnerabilities detected and measured (SAST), privilege activity monitoring, VPC/security group/firewall monitoring, encryption, and key rotation monitoring.
Expert Application functional and performance vulnerabilities detected and measured, real-time network traffic visibility, suspicious traffic detection, automated penetration testing (DAST/IAST), and cybersecurity integrated into product design and delivery.
Cutting-edge Production environments monitored and protected by a security engine, and auto-remediation in place.

Security & Compliance¶

Baseline Reactive security processes and manual security testing.
Novice Regular security audits, basic security training, and automated security testing.
Intermediate Proactive security processes, security as code, and security integrated into the pipeline.
Advanced Shift-left security and compliance, security champions, and automated security gates in the pipeline.
Expert Continuous security and compliance, security and compliance metrics, and automated security and compliance remediation.

Metrics¶

Baseline Cycle time and build frequency not measured, and no real-time feedback loops.
Intermediate Trend reports are manually generated, and stakeholders are notified of MTTR, change failure trend, and availability.
Advanced Cycle time and build frequency measured in real-time, automated trend reports, and team-wide notifications.
Expert Low MTTR, high availability, real-time dashboards for stakeholders, and real-time feedback loops.
Cutting-edge Cross-team data mining and analysis,
and business impact and performance connected to technical performance.

Infrastructure¶

Baseline Manual DR Active-Passive failover architecture, static VMs Infrastructure, manual patching, firewall and network management, and DNS record creation.
Intermediate Infrastructure as code in place for VMs and configuration management systems in place.
Advanced Environment standardization, immutable and ephemeral environments, automated deployments, cloud infrastructure for VMs (IaaS), automated patching, firewall and network management.
Expert Cloud infrastructure for containers (CaaS or PaaS), infrastructure as code for containers, zero downtime through cloud-based continuous operations, automated DNS record generation, and CDN in place.
Cutting-edge Continuous operations, serverless infrastructure, and traffic shaping in production environments.

Release Management¶

Baseline Infrequent releases with manual review and coordination, no end-to-end relationship between request and release.
Intermediate Partially automated releases requiring manual intervention.
Advanced Basic Dev-to-Ops traceability, velocity, and release feedback.
Expert Frequent releases with defined manual and automated orchestration and calendaring, full traceability from request to deployment.
Cutting-edge Just-in-time or on-demand releases, every change is deployed to production, and immediate feedback and triggers.

Risk Mitigation¶

Baseline Risk controls are done manually as part of the delivery process.
Intermediate Lean Change management controls in place and automated CR process.
Advanced Quality gates enforced during the CI phase, test controls enforced pre-release, and 100% test pass rate implemented.
Expert Security SAST gates enforced during the CI phase, DAST and AST enforced pre-release, database automation practices, and full audit trail of the delivery process.
Cutting-edge Security engine protects production state, compliance controls in place, proactive auto-healing and auto-scaling in production, and traffic shaping to ensure zero downtime deployments and reduced risk of outages.

Tools¶

Baseline Code build done on CI server like Jenkins using Maven and NPM, committing the code to a source control system like GitHub, and automated unit tests using JUnit and ESLint.
Intermediate Code scanning using SonarQube, functional tests using Rest Assured, Selenium, and Cucumber, artifact repository management using Nexus, configuration management using Ansible, collaboration tools like Confluence and Slack, and project management tools like Jira.
Advanced Static Application Security Testing (SAST) using Checkmarx, deploying VMs in GCP using Terraform, logging through Splunk, containerization (Docker), service discovery and version management using Consul, secret management using Vault, API mocking using Wiremock, database schema management using Flyway or Liquibase, dependency vulnerability check (OWASP), and contract test frameworks in place (Karate).
Expert Containerization orchestration using Kubernetes, AST/DAST using Contrast Security or OWASP ZAP, data logs visualization using Kibana, open-source monitoring using Prometheus, and performance test frameworks in place like JMeter.
Cutting-edge Service mesh using Istio, open-source data visualization using Grafana, service mesh observability and configuration using Kiali, feature toggles for A/B testing using LaunchDarkly or Intuit.

Innovation and Experimentation¶

Baseline Little to no innovation or experimentation, a risk-averse culture, and a strong focus on maintaining the status quo.
Novice Occasional innovation and experimentation initiatives, a culture that tolerates some risk-taking, and a moderate focus on exploring new ideas and approaches.
Intermediate Regular innovation and experimentation initiatives, a culture that encourages risk-taking and learning from failures, and a strong focus on exploring new ideas and approaches.
Advanced Systematic innovation and experimentation initiatives, a culture that embraces risk-taking and learning from failures, and a strong focus on continually pushing boundaries and seeking disruptive ideas.
Expert Organization-wide culture of innovation and experimentation, a strong focus on continuously pushing boundaries and seeking disruptive ideas, and a systematic approach to identifying, testing, and scaling new ideas and approaches.

Agility and Adaptability¶

Baseline Low agility and adaptability, a strong focus on following established processes and plans, and a resistance to change.
Novice Some agility and adaptability, occasional changes to processes and plans, and a moderate willingness to embrace change.
Intermediate Moderate agility and adaptability, regular changes to processes and plans, and a strong willingness to embrace change.
Advanced High agility and adaptability, continuous evolution of processes and plans, and a proactive approach to embracing change.
Expert Organization-wide culture of agility and adaptability, a strong focus on continuously evolving processes and plans, and a proactive approach to embracing change and seizing new opportunities.

By working towards continuous improvement in innovation and experimentation, agility and adaptability, organizations can better respond to market changes, customer needs, and new technological advancements. This enables them to stay competitive, deliver value more efficiently, and ultimately achieve better business outcomes. It is essential to create an environment where learning and growth are encouraged, and teams are empowered to take risks, learn from failures, and continuously improve their processes and approaches.

Collaboration and Communication¶

Baseline Limited collaboration and communication between departments, teams, or individuals, leading to a siloed working environment and reduced efficiency.
Baseline Siloed teams, limited cross-team communication, and a lack of transparency in decision making.
Novice Occasional collaboration and communication initiatives, some information sharing between departments, teams, or individuals, but still primarily operating in silos.
Novice Occasional cross-team communication and collaboration, some transparency in decision making, and basic information sharing.
Intermediate Regular cross-team communication and collaboration, increased transparency in decision making, and structured information sharing.
Intermediate Regular collaboration and communication initiatives, an emphasis on information sharing between departments, teams, or individuals, and efforts to break down silos.
Advanced A strong culture of collaboration and communication, proactive information sharing between departments, teams, or individuals, and the regular use of cross-functional teams.
Advanced Strong cross-team communication and collaboration, high transparency in decision making, and a culture of open information sharing.
Expert Seamless cross-team communication and collaboration, full transparency in decision making, and a strong culture of open information sharing and collective decision making.
Expert Organization-wide culture of collaboration and communication, seamless information sharing between departments, teams, or individuals, and the use of cross-functional teams to drive strategic initiatives.

Continuous Learning and Skill Development¶

Baseline Minimal focus on continuous learning and skill development, limited opportunities for professional development, and a lack of support for employee growth.
Novice Occasional focus on continuous learning and skill development, some opportunities for professional development, and moderate support for employee growth.
Intermediate Regular focus on continuous learning and skill development, multiple opportunities for professional development, and a strong commitment to employee growth.
Advanced A strong culture of continuous learning and skill development, extensive opportunities for professional development, and active support for employee growth.
Expert Organization-wide culture of continuous learning and skill development, a wide range of professional development opportunities, and a proactive approach to supporting employee growth and development.

By understanding these stages of maturity and working towards continuous improvement in each area, organizations can optimize their software delivery processes, enhance collaboration, and ultimately deliver better products and services. The key is to regularly assess and analyze the current state of maturity and identify areas for improvement, while also fostering a culture of learning and growth within the organization.

By fostering a culture of collaboration and communication, organizations can break down silos, improve information sharing, and drive better decision-making. In addition, by focusing on continuous learning and skill development, organizations can build a workforce that is adaptable and able to respond to evolving business needs.

By prioritizing these four dimensions, organizations can create an environment where continuous improvement is ingrained in the culture, and employees are empowered to innovate, adapt, collaborate, and learn. This approach will enable businesses to stay competitive in an ever-changing market and deliver long-term value to customers, stakeholders, and employees.