Jenkins : 2014 Jenkins Infrastructure Roadmap

Goals

• Consolidate primary project servers to the OSUOSL
  • WHY: Simplify server management, take advantage of shared resources/services offered by the OSUOSL.
• Migrate to Puppetmaster-based configuration management infrastructure
  • WHY: Gain inventory and status reporting from puppet-managed nodes, open up ability to easily bootstrap new servers, fix lack of orchestration and quality/delivery pipeline for Jenkins' own infrastructure
• Migrate key asset ownership to the SPI
  • WHY: Too much bus factor on domains and server ownership by Unknown User (rtyler)
• Upgrade core developer services, e.g. Confluence, JIRA.
  • WHY: Some of core developer services have reached an end-of-life status from their vendors, making the continued secure operation of those services impossible.
• Revamp infrastructure monitoring
  • WHY: Current monitoring infrastructure (Nagios-based) is not reliable and doesn't provide sufficient insight into the state of services, mostly just machines. More often downtime is noticed via Twitter/IRC rather than alerts.
• Clean-up/prune mirroring system
  • WHY: Primary mirrors (OSUOSL), and secondaries are not intended to be the binary archive for all historical releases and need to save disk space for other projects

Projects

Migration to a modern Puppet infrastructure

Owner: Unknown User (rtyler)

Phases of work

1. Set up Puppet Enterprise (Should be done in an automated fashion (no snowflakes!))
2. Create a continuous delivery pipeline in Jenkins for our infra
   1. Define a good testing/deployment workflow that Jenkins and Puppet Labs can point to and say "this is how things should be done in 2014+"
3. Add first production host to PE and start provisioning it via the CD pipeline
4. Blog about progress thus far
   1. Migration strategy
   2. Developer testing and deployment pipeline
   3. ??
5. Shift existing infrastructure over to the new model piece by piece
   1. Move primary/core services already managed by masterless Puppet, e.g. DNS, WWw, package repos
   2. Puppetize less-critical, more hacked together services (e.g. butlerbot, pieces of MirrorBrain)

Assigning ownership of key assets to SPI

Owner: Unknown User (kohsuke)

Core user/developer Service Upgrades

Owner: Unknown User (rtyler)

Phases of work

1. Puppetize JIRA installation
2. Puppetize Confluence installation
3. Migrate production JIRA
4. Migrate production Confluence

Mirror cleaning

Owner: Unknown User (kohsuke)

*COMPLETED*

Monitoring Revamp

Owner: Unknown User (rtyler)

Phases of work

1. Evaluation different monitoring solutions
   1. Sensu - overly complex for monitoring our small deployment
   2. collectd
   3. Nagios
2. Deploy/monitor one node
3. Roll out to rest of cluster

Jenkins JIRA: Components/workflow refactoring

Owners: Unknown User (danielbeck), Unknown User (oleg_nenashev)

More info: 2014 JIRA Components Refactoring

Jenkins IRC Bot Improvements

Owner: Unknown User (oleg_nenashev)

Phases of work

1. Unknown User (rtyler) - Puppetize IRC Bot => DONE
2. Unknown User (oleg_nenashev) - Describe the deployment flow on GitHub/Wiki => DONE
3. Unknown User (oleg_nenashev) - Introduce new commands (component management, etc.) => IN_PROGRESS
4. Unknown User (oleg_nenashev) - Improve IRC Bot's testability (local debug, test suites) => IN_PROGRESS