Small Teams Should focus on Dev not Ops š±
DevOps is a tricky concept. It make the two words seem equal but the goal of a small business or solo entrepreneur who is trying to build a product or internal application is not to spend half the budget on Ops.
Give it a listen here
NOTE: This is a very Laravel centric look but any modern framework should be able to offer all of this.
This article is for companies trying to build internal applications to streamline workflows. I will not suggest the many Saas products out there that make this drag and drop since for the most part they never really are customizable enough from what I see. But always consider your options and try it out at first it might be enough to see if it meets the business need your are trying to optimize or automate. This article will focus on those internal applications or saas product ideas that require one or more developers to build.
When starting a new project I get all of the Ops in place day one in less than two hours! Then I can focus on Dev not ops for the rest of the project.
This article will explain how to get ALL ops out of the way quick and easy so your developer(s) can focus on developing your product.
Just so you know my experience, I have been doing this 20+ years. For 10 of those years I built or helped build a lot of the infrastructure and workflows for Pfizer which are still in use for 100s of Laravel application on AWS (cause I had to use AWS and I was not able to use services like Forge š ). So I have seen many sides of the Dev and the Ops and now that I am back to working for smaller companies as a freelancer I see a lot of mistakes happening. Some of it is them trying to be to āenterpriseā and copying patterns from that are seen as āenterpriseā and others patters that just create complexity that lack any real data to back up the positive impact, if any, they have. Most if not all my suggestions come from books like Accelerate, Continuous Delivery and Dev Ops Handbook that I then have put into practice.
Here is an overview of the topics I will cover
- Hosting
- Storing your code
- Setting up Servers
- Local Development
- Continuous Integration and automation
- Deploying to Production and or Staging
- The Stack
- Tracking work
- Security
- Websockets
- Backups
- Day to Day
- File Hosting
- Logging
- Training and Documentation
- Scaling (if your idea makes it this far)
None of this is hard, with all the services I mention it is a lot of button clicking and reading docs.
Hosting
To start with do not use AWS š why? First and foremost it complicates things and the goal of this article will show how to keep things simple and grow as you need to grow with ease.
AWS has complexity and hidden cost risk (run away services etc) with very few benefits for a ānon-enterpriseā company eg the small business / team that this article is for.
Do not worry about scaling. Though everything I will suggest is easy enough to scale do not worry about it right now.
If you are a lone owner and are going to try and do this yourself then keep reading. If you are a lone owner or small business going to hire a person to build this then keep reading knowing that the person you hire will have their own ideas and tooling but that you can still learn from this article that it should not be hard or expensive or a big part of the project. If you are a developer then use this to your advantage.
I am going to focus on Digital Ocean for a number of reasons and each section will explain each choice.
Storing Your Code
Keeping it simple GitHub is nice because developers know it and it keeps their profile stats up so when they are done with your project they can show on their profile the fact they are pushing code to GitHub which matters in my opinion. You can also use GitLab or Bitbucket, but keep it simple.
Setting up Servers
The short answer is donāt! No this is not saying go serverless at all but donāt touch servers instead use Forge (links below) and with one click (maybe three!) you can do the following
- Setup a server
- Setup SSL (this is required and key and free!)
- Have queueing in place and ideally Horizon queuing in place
- Have scheduled commands in place
- SSR
- Have a logging UI in place
- Have a foundation to build on later if needed
- Have a cache system
So, 2ā3 clicks later, you have staging and production ready to go thanks to Forge. Forge is able to host Wordpress and other frameworks. Also I will talk later about using Forge to āscaleā when IF that time comes.
Local Development
As a developer you can use tools like HERD and Valet to get going locally. Keep it simple. Your local does not need to mimic production as long as you have the correct version of PHP you are pretty good to go. Do not use Docker or Sail UNLESS you need to. Like if you are on Windows for example then use use those great tools. Or maybe you like to š¤·š»āāļø but keep it simple and none of this should take time from development.
Using Laravel Docs as a base line of how to get started, coding patterns and more makes bringing on new developers easier then you trying to build a lot of custom processes and patterns.
Continuous Integration and automation
As the day goes on and you are pushing code the CI system should be using tools like Pint and PHPStan (both supported by Laravel maintainers) check what is pushed.
This way, you and your small team can easily push code to trunk or a pull request and know that the required patterns are in place and that PHPStan has your back. On top of that a sufficient suite of tests, say 50ā70% coverage but 90% confidence, is also helping to know the push is āgood enoughā to keep things moving along. There are lots of articles, books and videos on this like the book Continuous Delivery and Accelerate have a lot of data on this as well.
The bottom line is CI takes less than 10 minutes to setup, 2ā5 minutes to run and should not be something you are thinking about after setup except when it saves your bacon. And when it fails do not blow it off. Most likely it found a real mistake.
Deploying to Production and or Staging
This again I keep simple, using the Envoy.io service I can integrate with Forge to deploy with āzeroā downtime to production and staging. Every successful merge or push into trunk will run this. On most projects that I do that do not have payment systems or a reason to worry about a process like sending emails I just have a production site no staging. For example one project sends emails to customers and another sms messages and I have staging for that so product owners and users can QA there without sending real messages. My builds push to both staging and production for example.
Using Laravel Pennant, I use feature flags to keep features off in production if that is required. I do not do complex branching patterns per the book Accelerate and Continuous Delivery book and videos that talk about the benefits to focusing on a trunk based workflow.
The Stack
In this case every stack has the same setup so that all projects are the same. The tech is not so important as much as consistency. I will not mention the frontend stack since it does not matter here.
- Laravel
- Pusher and websockets since most projects I do need this but the Laravel docs do suggest some open source options.
- Queues with Horizon since all projects I do need this for sooooo many things
- Log Viewer https://laravel-news.com/beautiful-log-viewer-for-laravel this is an example of a consistent way to do this.
- Sentry or some system to help with deployed application errors
- Pennant for Feature Flags this is key for getting code to production daily.
- Ziggy so routes are not hard coded
- Pint
- LaraStan
- Spatie Data objects so the team can have a consistent way to pass object into and out of classes
- JetStream so out of the box I have 2fa, profile pages etc taken care of.
There are many others but a good foundational install can get any project going quickly. Github has the idea of a Template repo so you can just get started with a new project from this foundation.
Tracking work
If you want to focus on developing then do not make this hard. I tried so many systems and in the end Trello stands out since it quickly allows me to add tickets share with the customer etc. There is no need for complex kanban boards or Gantt charts or other ui features. I tried ClickUp for a while but making a ticket became way too many clicks imo. So any system that is simple, and can communicate these things:
- What is in progress
- What needs QA
- What is next
- What is later
Security
Focusing on Dev and not ops can be tricky when it comes to this but letās list out some simple goals.
Keep the stack up to date.
GitHub will help with this with itās automated security patching and pushing to main and therefore staging and production so this is great. Use services like Laravel Shift to save time on the MAJOR updates.
Servers are pretty easy to replace with Forge and a good build process.
Make Servers easy to update
Using DigitalOcean Database Droplets your Web Server can be replaced with a new version of Ubuntu in no time so a security update does not have to be tedious. I will talk about remote files in a moment that also make that server less important as well. With these items in place you can keep the system up to date.
Using 2fa on the site you have some security there as well. And of course best practices when writing code.
Using Laravel Jetstream makes this a breeze https://jetstream.laravel.com/introduction.html
Some people will use Cloudflare as well as a WAF (Web Application Firewall) which is great just keep it simple.
Websockets
The Laravel Docs are good on helping to set this up. For now I use a service called Pusher.com and for 10 years it has just worked. The nice thing here is your application can be truly dynamic for the user like Chat widgets. For example one project a customer does a long running import and when it is done I have the system respond using Laravel Broadcast so the ui updates in real time avoiding the cost of polling.
Day to Day
I see lotās of companies complicate the day to day with Standups and what not. In my opinion, especially for a small team, keep it simple. Monday have a check in with goals for the week. End of day have developers just put in Slack what they worked on today, what is up for tomorrow and if any blockers. Make it clear people should reach out for help as needed. Using a 1 hour timebox as a way to give space to try and figure something out before reaching out just not longer. If a build is broke on truck that person must fix it in 20 minutes or someone can roll back their work. If there is a Pull Request to review do not DM a room but DM a person and give them 20 minutes to get it done or find someone else. One person per PR (if you even bother with a PR flow links below with videos on why this is not a required idea).
37Signals has books and a podcast about this stuff. Not thinking in 2 week sprints, building measure learn, 6 week cycles etc.
Build trust and confidence in each other knowing that mistakes happen no matter the process.
File Hosting
Digital Ocean Spaces mimics the AWS S3 api so it is easy to keep files there and not on the server. This makes the replacement of a server much easier (security updates, load balancing etc)
If your application has no need for this, like all they do is fill in forms or save files for temporary usage then it is one more thing not to bother with. But starting out with this is not a bad idea IF the system needs to store files long term. Really needs to.
Logging
There are services like PaperTrail and others that can help get logs off the server. This is not bad since if there is a security issue you have a backup of logs. But for the most part this can wait and just use the ui tool suggested above and Sentry like service to catch errors and notify via Slack.
Training and Documentation
Once you establish patterns and processes then put them in a Google Doc or the Readme.md file at the root of the projectās code. This way any new developer can hop in and go. It is not something you do later but as you go. Your CI system in someways documents setup since it does it every build. You might have local seeders then note that in the README.md. Basically do what you can so that any new person can get going with little to no help.
Backups
Do them! Forge has a way to backup databases so turn it on and try to recover locally once a month to make sure they are good. Files do not need to be backed up since your code is in Github. If customers have files that matter then put them on DigitalOcean Spaces and make sure revisions are on there. Make another Space for an extra backup if needed and tar files over there every day using https://spatie.be/docs/laravel-backup/v8/introduction so you can know the files are in an extra place. Consider this later if the product actually gets use.
Scaling (if your idea makes it this far)
Lastly scaling. If your project makes it this far congrats. If it needs scaling then there is no need for Kubernetes or Docker or Node/Python Serverless. Some simple things you can do:
- Using a Load balancer, built into Forge, have 2 or more web servers taking the traffic.
- If it is long running jobs like a big import use Queues to spread out the work with Horizon even having a Queue focused server
- If one route is getting massive amounts of traffic consider putting that one on Vapor.Laravel.com and have it then put the results in a Queue the main system can pick up.
- Redis is there to cache so start putting more request into cache as needed
The point is that if you get to this level then āmovingā to a solution is not something you have to fear from the start since all the above leave you in a place that it it is easy to extend into these areas. It is not a big shift.
Donāt be me
I really enjoy coding and have spent too many years dong ops stuff. People who like Ops typically are not the same people who like to code. Both are amazing skills but if you want to build an application for your customers or your staff to get their work done more quickly or you have an amazing idea that you want to build then focus on the Dev part leaving ops to be done and forgotten in less than 2 hours at the start of the project. š
Want help? Reach out to me at https://www.linkedin.com/in/alfrednutile/
Links
12 Factor App https://12factor.net/
Forge https://forge.laravel.com/
Accelerate https://www.amazon.com/Accelerate-Software-Performing-Technology-Organizations/dp/1942788339
Continuous Delivery https://www.youtube.com/@ContinuousDelivery
Dev Ops Handbook https://www.amazon.com/DevOps-Handbook-Second-World-Class-Organizations
LaraStan https://github.com/nunomaduro/larastan
Pint https://laravel.com/docs/10.x/pint
Github CI/CD actions link https://github.com/alnutile/course-zero-to-production/blob/main/.github/workflows/ci-cd.yml
Laravel Shift https://laravelshift.com/
Anti PR Flow video https://youtu.be/ASOSEiJCyEM?si=51FKtzDM30t9ZlBv
Laravel Herd https://herd.laravel.com/
37Signals https://37signals.com/podcast/