I used to have to use a CI pipeline at work that had over 40 jobs and 8 stages for checking some sql syntax and formatting and also a custom python ETL library that utilized pandas and constantly got OOM errors.
They didn’t write any unit tests because “we can just do that in the CI pipeline” and if you didn’t constantly pull breaking changes into your branch you would guarantee the pipeline would fail, but if you were lucky you only had to restart 30% of your jobs.
It was the most awful thing and killed developer productivity to the point people were leaving the team because it sucks to spend 40% of your time waiting for CI scripts to fail while you are being yelled at to deliver faster.
I used to have to use a CI pipeline at work that had over 40 jobs and 8 stages for checking some sql syntax and formatting and also a custom python ETL library that utilized pandas and constantly got OOM errors.
They didn’t write any unit tests because “we can just do that in the CI pipeline” and if you didn’t constantly pull breaking changes into your branch you would guarantee the pipeline would fail, but if you were lucky you only had to restart 30% of your jobs.
It was the most awful thing and killed developer productivity to the point people were leaving the team because it sucks to spend 40% of your time waiting for CI scripts to fail while you are being yelled at to deliver faster.