Tell future-you why you did that thing; they can read but don't know what you intended. Oral tradition is like never writing state to disk; flush those buffers.

Platform engineering is trying to deliver the self-service tools teams want to consume to rapidly deploy all components of software. While it may sound like a TypeScript developer would feel more empowered by writing their infrastructure in TypeScript, the reality is that it’s a significant undertaking to learn to use these tools properly when all one wants to do is create or modify a few resources for their project. This is also a common source of technical debt and fragility. Most users will probably learn the minimal amount they need to in order to make progress in their project, and oftentimes this may not be the best solution for the longevity of a codebase. These tools are straddling an awkward line that is optimized for no-one. Traditional DevOps are not software engineers and software engineers are not DevOps. By making infrastructure a software engineering problem, it puts all parties in an unfamiliar position. I am not saying no-one is capable of using these tools well. The DevOps and software engineers I’ve worked with are more than capable. This is a matter of attention. If you look at what a DevOps engineer has to deal with day-in and day-out, the nuances of TypeScript or Go will take a backseat. And conversely, the nuances of, for example, a VPC will take a backseat to a software engineer delivering a new feature. The gap that the AWS CDK and Pulumi try to bridge is not optimized for anyone and this is how we get bugs, and more dangerously, security holes.

Get production-accurate data and preview databases to code against, fast.

Use ssh -f -N -L to run the port-forwarding session in the background.

一个成熟的自动化运维系统具备什么功能？

有哪些好用的devops工具，适合中小企业开发？

If we say that MLOps is just DevOps + “some things”, then CI/CD is a core principle of that.

DevOps is a set of practices, tools, and a cultural philosophy that automate and integrate the processes between software development and IT teams. It emphasizes team empowerment, cross-team communication and collaboration, and technology automation.

In a Staging workflow, releases are slower because of more steps, and bigger because of batching.

For Staging to be useful, it has to catch a special kind of issues that 1) would happen in production, but 2) wouldn’t happen on a developer's laptop.What are these? They might be problems with data migrations, database load and queries, and other infra-related problems.

DevOps is an interesting case study for understanding MLOps for a number of reasons: It underscores the long period of transformation required for enterprise adoption.It shows how the movement is comprised of both tooling advances as well as shifts in cultural mindset at organizations. Both must march forward hand-in-hand.It highlights the emerging need for practitioners with cross-functional skills and expertise. Silos be damned.

What is End User Computing (EUC)? Thanks to the progressive introduction of DevOps, attention to the role of the end user in software development and testing has increased significantly. It is now important to think like an end user when we develop and test software. After all, that's what we're all doing it for.

you can also mount different FastAPI applications within the FastAPI application. This would mean that every sub-FastAPI application would have its docs, would run independent of other applications, and will handle its path-specific requests. To mount this, simply create a master application and sub-application file. Now, import the app object from the sub-application file to the master application file and pass this object directly to the mount function of the master application object.

There are officially 5 types of UUID values, version 1 to 5, but the most common are: time-based (version 1 or version 2) and purely random (version 3). The time-based UUIDs encode the number of 10ns since January 1st, 1970 in 7.5 bytes (60 bits), which is split in a “time-low”-“time-mid”-“time-hi” fashion. The missing 4 bits is the version number used as a prefix to the time-hi field.  This yields the 64 bits of the first 3 groups. The last 2 groups are the clock sequence, a value incremented every time the clock is modified and a host unique identifier.

Even though most posts are warning people against the use of UUIDs, they are still very popular. This popularity comes from the fact that these values can easily be generated by remote devices, with a very low probability of collision.

Artifactory/Nexus/Docker repo was unavailable for a tiny fraction of a second when downloading/uploading packagesThe Jenkins builder randomly got stuck

Microservices can really bring value to the table, but the question is; at what cost? Even though the promises sound really good, you have more moving pieces within your architecture which naturally leads to more failure. What if your messaging system breaks? What if there’s an issue with your K8S cluster? What if Jaeger is down and you can’t trace errors? What if metrics are not coming into Prometheus?

If you’re going with a microservice:

Let’s take a simple online store app as an example.

some of the pros for going microservices

We’ve spent a lot of times telling our developers, go faster, go faster. And then turning to our operators and say, keep it stable. And I’ve been that operator. I know the best way to keep the system stable is to not change it. But that’s not good for our customers.

In contrast, a defining feature of ML-powered applications is that they are directly exposed to a large amount of messy, real-world data which is too complex to be understood and modeled by hand.

Upcoming Trends in DevOps and SRE in 2021 DevOps and SRE are domains with rapid growth and frequent innovations. With this blog you can explore the latest trends in DevOps, SRE and stay ahead of the curve.

This means that an event-driven system focuses on addressable event sources while a message-driven system concentrates on addressable recipients. A message can contain an encoded event as its payload.

we want systems that are Responsive, Resilient, Elastic and Message Driven. We call these Reactive Systems.

Resilience is achieved by replication, containment, isolation and delegation.

Today applications are deployed on everything from mobile devices to cloud-based clusters running thousands of multi-core processors. Users expect millisecond response times and 100% uptime. Data is measured in Petabytes.

It basically takes any command line arguments passed to entrypoint.sh and execs them as a command. The intention is basically "Do everything in this .sh script, then in the same shell run the command the user passes in on the command line".

#!/bin/bash
set -e

... code ...

exec "$@"

We should think about the number of simultaneous connections (peak and average) and the message rate/payload size. I think, the threshold to start thinking about AnyCable (instead of just Action Cable) is somewhere between 500 and 1000 connections on average or 5k-10k during peak hours.

As it stands, sudo -i is the most practical, clean way to gain a root environment. On the other hand, those using sudo -s will find they can gain a root shell without the ability to touch the root environment, something that has added security benefits.

Much like sudo su, the -i flag allows a user to get a root environment without having to know the root account password. sudo -i is also very similar to using sudo su in that it’ll read all of the environmental files (.profile, etc.) and set the environment inside the shell with it.

This means that unlike a command like sudo -i or sudo su, the system will not read any environmental files. This means that when a user tells the shell to run sudo -s, it gains root but will not change the user or the user environment. Your home will not be the root home, etc. This command is best used when the user doesn’t want to touch root at all and just wants a root shell for easy command execution.

Though there isn’t very much difference from “su,” sudo su is still a very useful command for one important reason: When a user is running “su” to gain root access on a system, they must know the root password. The way root is given with sudo su is by requesting the current user’s password. This makes it possible to gain root without the root password which increases security.

“su” is best used when a user wants direct access to the root account on the system. It doesn’t go through sudo or anything like that. Instead, the root user’s password has to be known and used to log in with.

Different data sources are better suited for different types of data transformations and provide access to different data quantities at different freshnesses

MLOps platforms like Sagemaker and Kubeflow are heading in the right direction of helping companies productionize ML. They require a fairly significant upfront investment to set up, but once properly integrated, can empower data scientists to train, manage, and deploy ML models. 

…Well, deploying ML is still slow and painful

DevOps Services

Automation suggests that a sysadmin has invented a system to cause a computer to do something that would normally have to be done manually. In automation, the sysadmin has already made most of the decisions on what needs to be done, and all the computer must do is execute a "recipe" of tasks. Orchestration suggests that a sysadmin has set up a system to do something on its own based on a set of rules, parameters, and observations. In orchestration, the sysadmin knows the desired end result but leaves it up to the computer to decide what to do.

For instance, automation usually involves scripting, often in Bash or Python or similar, and it often suggests scheduling something to happen at either a precise time or upon a specific event. However, orchestration often begins with an application that's purpose-built for a set of tasks that may happen irregularly, on demand, or as a result of any number of trigger events, and the exact results may even depend on a variety of conditions.

Tabular Comparison Between All Deployment Methods:

more developers are becoming DevOps skilled and distinctions between being a software developer or hardware engineer are blurring

Secrets management is one of the most sensitive and critical disciplines in all of DevOps and is becoming increasingly important as we move toward a fully continuous deployment world. AWS Keys, deploy keys, ssh keys are often the key attack vector for a bad actor or insider threat, and thus all users and customers are concerned about robust secrets management.

In some contexts, "ops" refers to operators. Operators were the counterparts to Developers represented in the original coining of the term DevOps.

Continuous Delivery of Deployment is about running as thorough checks as you can to catch issues on your code. Completeness of the checks is the most important factor. It is usually measured in terms code coverage or functional coverage of your tests. Catching errors early on prevents broken code to get deployed to any environment and saves the precious time of your test team.

Continuous Integration is a trade off between speed of feedback loop to developers and relevance of the checks your perform (build and test). No code that would impede the team progress should make it to the main branch.

A good CD build: Ensures that as many features as possible are working properly The faster the better, but it is not a matter of speed. A 30-60 minutes build is OK

A good CI build: Ensures no code that breaks basic stuff and prevents other team members to work is introduced to the main branch Is fast enough to provide feedback to developers within minutes to prevent context switching between tasks

The idea of Continuous Delivery is to prepare artefacts as close as possible from what you want to run in your environment. These can be jar or war files if you are working with Java, executables if you are working with .NET. These can also be folders of transpiled JS code or even Docker containers, whatever makes deploy shorter (i.e. you have pre built in advance as much as you can).

Continuous Delivery is about being able to deploy any version of your code at all times. In practice it means the last or pre last version of your code.

Continuous Integration is not about tools. It is about working in small chunks and integrating your new code to the main branch and pulling frequently.

The app should build and start Most critical features should be functional at all times (user signup/login journey and key business features) Common layers of the application that all the developers rely on, should be stable. This means unit tests on those parts.

Continuous Integration is all about preventing the main branch of being broken so your team is not stuck. That’s it. It is not about having all your tests green all the time and the main branch deployable to production at every commit.

While talking about DevOps, three things are important continuous integration, continuous deployment, and continuous delivery.

Basic prerequisites to learn DevOps

DevOps benefits

Operations in the software industry include administrative processes and support for both hardware and software for clients as well as internal to the company. Infrastructure management, quality assurance, and monitoring are the basic roles for operations.

I set it with a few clicks at Travis CI, and by creating a .travis.yml file in the repo

language: node_js
node_js: node

before_script:
  - npm install -g typescript
  - npm install codecov -g

script:
  - yarn lint
  - yarn build
  - yarn test
  - yarn build-docs

after_success:
  - codecov

I set it with a few clicks at Travis CI, and by creating a .travis.yml file in the repo

language: node_js
node_js: node

before_script:
  - npm install -g typescript
  - npm install codecov -g

script:
  - yarn lint
  - yarn build
  - yarn test
  - yarn build-docs

after_success:
  - codecov

Continuous integration makes it easy to check against cases when the code: does not work (but someone didn’t test it and pushed haphazardly), does work only locally, as it is based on local installations, does work only locally, as not all files were committed.

With Codecov it is easy to make jest & Travis CI generate one more thing:

Continuous Deployment is the next step. You deploy the most up to date and production ready version of your code to some environment. Ideally production if you trust your CD test suite enough.

DevOps has taught us that the software development process can be generalized and reused for dealing with change not just in application code but also in infrastructure, docs and tests. It can all just be code.

Environment variables are 'exported by default', making it easy to do silly things like sending database passwords to Airbrake.

Valdomiro Bilharvas - Squads mais eficientes com Devops

Daniel Wildt, Guilherme Lacerda - Voltando para as raízes do Desenvolvimento Ágil

João Brito - CI/CD - Pense um pouco além das ferramentas

Allan Moraes - Automatizando o Monitoramento de Infraestrutura

Amanda Matos - Métricas & DevOps - Por que você deve medir para conquistar?

Aurora Li Min de Freitas Wang - Sendo dev em meio ao DevOps: Mudança de cultura de baixo para cima

Mateus Prado - DevOps Engineers: por que é tão difícil contratar?

Tiago Roberto Lammers - Nossa jornada DevOps na Delivery Much para microserviços e o que aprendemos

Program

Acabe com o caos no seu pipeline com 4 ferramentas de métricas e controle

Encerramento e Sorteios

Hashicorp Vault: One-Time Password para SSH

Como o iFood criou seu próprio RDS?

Repositorio NPM privado grátis com Verdaccio e AWS

Usando Traefik para automatizar o proxy reverso de seus containers docker

Como construir uma stack ELK sem dor de cabeça

Pipeline de CI/CD no Kubernetes usando Jenkins e Spinnaker

Implementando CI com GitLab

Trilha DevOps Tools

Jenkins Pipelines

Grafana

Prometheus

Intervalo para Almoço

Você consegue visualizar a saúde da sua aplicação?

We have also begun developing an instrument to assess organizations' readiness to adopt Agile and DevOps. We would welcome opportunities to pilot this assessment instrument with your organization.

There is only one codebase per app, but there will be many deploys of the app

Why Pipeline?

at its core, DevOps is about culture

The true essence of DevOps is empathy.

Is it because ops care deeply about systems while devs consider them a tool or implementation detail?

When I look at the DevOps “community” today, what I generally see is a near-total lack of overlap between people who started on the dev side and on the ops side.