Best practice for test and production environments

I've worked in many companies, each one with a different set of environments, the one I have like the most had 5 environments:

1) Local: Basically your machine. Here is where you code and test your changes before even asking a peer for review.

2) Dev: If for some reason you cannot test your code locally (Dependencies issues mainly: "My code has neves compiled in my machine but it compiles perfectly in Jenkins/Bamboo/Travis"), then you push your changes to your feature branch in Git and make Bamboo compile it and deploy it to a dev server where you can test (you are still not sure if it will work, so no peer review so far).

3) Staging: You think your code work and you love how it looks like. You create a Pull Request in order for you peers to take a look at it before it gets merged to the master branch. Here they make comments and you fix possible issues, since you are more sure about your changes you make Bamboo deploy it to the Staging environment where more "stable" code lives and more realistic data is stored in whatever database is there. Once deployed another developer/tester can check your changes actually work and make you a "QA Sign off in Staging Env."

4) Stable: Ok, now you are ultimate sure your changes work since you deployed to Staging and nothing got broken. You merge your branch to master and Bamboo compiles master and deploys to another sets of servers in a Stable Environment (no one else should merge to master until you finish your deployment to Production, to avoid a merge of not related merges). This environment should be a replica from Production, data, code and server conditions. Here is where you show your changes to your manager, product owner or the person in charge to validate your work before sending it to production. You get the final approval, everything is good, you are sweaty, you have worked for 30 days in a row to get this change done, your wife divorced you but you are very confident it works perfectly.

5) Production: Where the clients connect to consume the company services, or the final build of your software to send to customers. With a few clicks in Bamboo you make it deploy to Production servers or compile the final build. It is green, everything seems to be ok. You check Splunk looking for errors, everything is good, life is good, you drink another sip of coffee before leaving, you drive home and sleep all weekend with your dog by your side.

It's a happy ending because having so many "Test" environments ensure quality, no change gets to production until EVERYBODY (not just you) are completely sure that changes is working.

I believe in order to establish an environment strategy that supports all ALM/SDLC activities 4 requirements should exist:

1) Development environment: to allow Dev to play around with new code/concepts and typically unit test with some basic integration testing using stubs and drivers. This environment should have loose change control procedures and would typically not be anywhere near the same scale as production. This is where the Dev team can build and tear down setups as required.

2) Interop environment: where integration of systems can be further tested and increased capability for non-functional testing I.e might be a resilient env with greater scalability than Dev. I'd see this environment having tighter change control and management. Test would perform integration and system testing in this environment.

3) Reference Architecture: This is what some might call pre-prod but is essentially the same as production in terms of scale and resilience. This would have change control and management procedures akin to prod. This env would support further test activities especially full scale performance testing, failover as well as operational fault triage and maintenance activities once a product is launched to customers.

4) Production: This environment will support live customers and so test activities will be limited once this is the case. This will be fully managed and have strict change management and config management processes.

Hope this helps

At our company there are two databases too, a test and a production database. The test database is mainly used for testing by developers but sometimes for business tests too. This database is refreshed daily using an actual copy of the production database. So this database can be both a playground and a serious testing database. But a third, development, database is the best option. We had one, but it is broken at the moment. But when you get one of those, you should make sure it is refreshed often enough. When developers use it as a playground, it will stray away from the production environment, and its data will be both old and currupt. Because of this, developers won't be able to test well themselves. So make sure you refresh this database periodically (maybe daily too, or at least once a week).

If possible, give each developer their own database on their local machine. That way they can do whatever they want with it without affecting anyone else. This should significantly decrease the desire to play with the test database, providing a more stable environment for business UAT.