AI Tooling

Learn the operating system for AI development

Agents, skills, planning, branching, and quality gates explained as one system for building faster without losing project direction.

Operating model

Plan
01

Shape intent into specs, non-goals, and verification criteria.

Context
02

Load repo rules, architecture notes, and task-specific memory before edits.

Execute
03

Use agents and skills on scoped feature branches with focused checks.

Promote
04

Move through dev, test, and prod only after checks and review.

10 topics found

agentsorchestration

Orchestration

Managed agent API adoption

OpenAI's Agents API entered public beta on September 10, 2026 as a managed Codex harness for durable cloud agents. It manages agent sessions, orchestration, context compaction, and recovery while the application still chooses tools and an execution environment. Adopt that split deliberately: the API can run the agent, but your application still owns user authorization, business state, side-effect receipts, retention decisions, and release gates.

agents-apimanaged-harnesssessionsproduction

Principles

  • Treat the managed harness as an execution dependency, not as the system of record for business workflow or authorization.
  • Choose hosted, self-hosted, partner, or no environment from data boundaries, required capabilities, isolation, latency, and operational ownership.
  • Record session, environment, artifact, and external side-effect identifiers in application state so retries and audits do not depend on a transcript.
  • Confirm retention and residency requirements before adoption; a self-hosted sandbox does not make the current Agents API eligible for Zero Data Retention.

Steps

  • Map the workload's files, tools, secrets, network paths, side effects, latency target, and retention obligations before choosing an environment.
  • Define a versioned session contract with instructions, model, tool allowlist, environment, concurrency limit, budgets, and terminal outcomes.
  • Keep end-user authorization, approvals, idempotency records, and business workflow transitions in the application control plane.
  • Consume session events into durable application records, correlate artifacts and tool effects, and make resumed work conditional on the latest accepted state.
  • Run outcome, recovery, isolation, retention, and cost evaluations before a canary; roll back to the prior execution path on a material regression.

Avoid

  • Assuming API-managed recovery proves an external write did or did not happen.
  • Selecting a self-hosted sandbox while ignoring that session state still passes through the managed API.
  • Enabling subagents without a workload-specific concurrency budget or shared-filesystem conflict plan.
  • Treating public beta behavior, limits, or data controls as a permanent production contract.
  • Giving the agent broad tools or vault access because the runtime itself is managed.
qualitypractice

Quality

Agent data-access boundaries

A coding agent can receive information through repository files, editor context, terminal output, and connected services. Content exclusion, MCP server admission, tool availability, execution permission, and sandboxing constrain different parts of that path. GitHub's September 2, 2026 release brings content exclusions to the Copilot app and CLI for Business and Enterprise customers; it does not make every agent surface equivalent. Audit the exact client, mode, identity, and data path before trusting a policy label.

agentsdata-accesscontent-exclusionmcppolicy

Principles

  • Document policy-surface coverage: one enabled setting is not proof of enforcement across every editor, terminal, and hosted agent.
  • Keep content exclusions separate from filesystem access and downstream service authorization; an admitted integration still needs narrowly scoped credentials.
  • Treat tool availability as a selection boundary and execution permission as an invocation boundary. Neither replaces the other.
  • Use synthetic evidence and preserve unknowns instead of opening protected files or declaring universal protection.

Steps

  • Inventory sensitive data classes and all routes into agent context without reading protected payloads.
  • Build a client-and-mode matrix with policy owner, effective settings, provider support, verification date, and evidence for each data path.
  • For Copilot, check documented indirect editor-context, symlink, and remote-filesystem limitations; follow the documented policy refresh procedure before testing.
  • Inspect managed MCP endpoint or command matchers, deny conflicts, and credential scope. Do not use a renamable server label as a security identity.
  • With test authorization, pair an allowed synthetic fixture with a denied one and inspect enforcement evidence. A missing marker in the final answer alone is not proof of denial.
  • Record confirmed gaps, untested surfaces, a responsible owner, and a retest plan using Agent Data Access Audit. Stop testing on unexpected exposure.

Avoid

  • Assuming the September app/CLI release implies support in every editor agent mode.
  • Confusing similarly named registry policies and enterprise managed MCP settings.
  • Treating a configured allowlist as proof that every tool on an admitted server is authorized.
  • Using private code, real tokens, or customer records as audit fixtures.
  • Silently weakening policy or retrying through another tool after a denial.
agentsfoundation

Foundation

What agents are

Agents are tool-using AI workers that can inspect a project, decide which tools to call, make bounded changes, run checks, and iterate toward a defined outcome. Treat them like capable teammates with a clear task contract: they need context, allowed actions, verification requirements, and a stopping point so their speed serves the project instead of drifting away from it.

agentstoolsautonomycontext

Principles

  • Treat agents as bounded executors, not independent owners.
  • Give agents clear inputs, allowed tools, verification gates, and stop conditions.
  • Keep durable project rules outside chat so every agent starts from the same contract.

Steps

  • Define the outcome in one concrete sentence.
  • Load the project contract and only the docs relevant to the task.
  • Let the agent inspect before editing.
  • Require verification output before accepting work.

Avoid

  • Letting the agent infer product direction from a vague prompt.
  • Skipping context load because the change looks small.
  • Accepting a diff without checking the behavior it changes.
skillspractice

Practice

When to build skills

Skills are worth building when the same agent behavior keeps showing up across projects, repos, or teams and the workflow has rules stable enough to document. A good skill packages triggers, constraints, examples, scripts, references, and expected outputs so an agent can repeat the work reliably without rediscovering the process every time.

skillsreusepromptsautomation

Principles

  • Build a skill when the workflow repeats and has stable rules.
  • Keep skill instructions specific enough to prevent drift but short enough to be followed.
  • Prefer scripts and templates inside skills when repetition is mechanical.

Steps

  • Name the workflow the skill owns.
  • Write trigger conditions and non-goals.
  • Add command recipes, references, and example outputs.
  • Test the skill on a real repo task before treating it as standard.

Avoid

planningpractice

Practice

Planning without drift

Planning without drift means turning loose ideas into execution-ready specs before agents start changing the codebase. The plan should preserve product intent through scope, non-goals, acceptance criteria, branch naming, verification gates, and review notes, giving every contributor the same target even when implementation moves quickly.

planningspecsscoperoadmap

Principles

  • Use brainstorms for discovery and specs for execution.
  • Approve scope before implementation begins.
  • Record non-goals as aggressively as goals.

Steps

  • Capture the idea in Plan or the planning source of truth.
  • Promote only clear work into an approved spec.
  • Create a feature branch named after the spec.
  • Verify against acceptance criteria before promotion.

Avoid

  • Treating a brainstorm as an implementation contract.
  • Adding adjacent features while already inside a feature branch.
  • Skipping non-goals because the desired outcome feels obvious.
workfloworchestration

Workflow

Feature to prod cycle

A clean feature-to-prod cycle separates local implementation from integration, staging validation, and production release decisions. Feature branches contain focused work, `dev` absorbs reviewed integration, `test` proves the release candidate, and `prod` becomes the intentional deployment boundary instead of whatever branch happened to build successfully.

branchesdeploymentvercelrelease

Principles

  • Feature branches isolate implementation.
  • `dev` is for integration, `test` is for release validation, and `prod` is the production promotion branch.
  • Production deploys should come from a known release branch, not an arbitrary local state.

Steps

  • Branch from the current integration base with `feature/<slug>`.
  • Open a PR from feature to `dev` after lint, build, and review.
  • Promote `dev` to `test` for staging validation.
  • Promote `test` to `prod` for production deployment.
  • Tag the release or merge back to the long-lived default branch after production is healthy.

Avoid

  • Deploying production directly from a feature branch.
  • Letting `dev`, `test`, and `prod` diverge for long periods.
  • Promoting without rerunning checks at each boundary.
workfloworchestration

Orchestration

Orchestrating large work

Large-scale AI development works when the effort is split into small, independently verifiable slices that share one source of truth. Specs define the target, Brain captures durable discoveries, agents execute scoped branches, and handoffs summarize behavior and risks so parallel work compounds instead of creating incompatible assumptions.

orchestrationmulti-agentsliceshandoff

Principles

  • Split broad work into independently verifiable slices.
  • Give each agent one slice and one source of truth.
  • Use Brain for durable discoveries and Plan for execution scope.

Steps

  • Start from an approved spec.
  • Create slices by boundary, risk, or user-facing workflow.
  • Run agents on separate feature branches when work can conflict.
  • Merge only after checks and a human-readable summary of shipped behavior.

Avoid

  • Parallelizing before the architecture boundary is clear.
  • Letting each agent invent its own acceptance criteria.
  • Using chat history as the only handoff artifact.
qualityorchestration

Quality

Quality gates for AI work

AI-assisted changes need explicit quality gates because fast implementation can hide broken behavior, weak assumptions, or undocumented drift. Baseline lint and build checks catch obvious failures, while browser verification, review, preview deployments, and durable notes make sure the shipped result matches the intended workflow instead of merely compiling.

verificationreviewtestingci

Principles

  • Make the smallest check mandatory and broader checks conditional on risk.
  • Review behavior, not just generated code.
  • Capture failed commands when they reveal repeatable traps.

Steps

  • Run focused checks while implementing.
  • Run the repo baseline gate before review.
  • Use preview deployments for user-facing routes.
  • Promote durable lessons into Brain or docs after closeout.

Avoid

  • Assuming generated code is correct because it compiles.
  • Letting UI changes ship without a browser check.
  • Forgetting to update docs when workflow surfaces change.
qualityorchestration

Quality

Eval-driven agent development

Eval-driven agent development turns product requirements and real failures into executable tasks before agent behavior is changed. Strong suites isolate every trial, grade observable outcomes with deterministic checks, use calibrated model judges only for subjective qualities, repeat important tasks to measure reliability, and preserve traces so regressions become reproducible engineering work instead of arguments about whether an agent merely feels better or worse.

evalsagentstracingci

Principles

  • Evaluate the model, harness, tools, environment, and policy as one versioned system.
  • Grade end-state outcomes with code whenever the requirement is directly observable.
  • Keep capability experiments separate from high-reliability regression gates.

Steps

  • Convert a real product failure into a task with clean initial state and explicit success criteria.
  • Run independent repeated trials and capture the full configuration plus trace.
  • Combine deterministic outcome graders with calibrated rubric graders where judgment is unavoidable.
  • Compare the candidate with a pinned baseline and block promotion on material regressions.

Avoid

  • Grading only the final response while ignoring whether the environment actually changed.
  • Requiring one exact tool path when several safe solutions can produce the correct outcome.
  • Letting answer keys, shared state, or infrastructure differences contaminate the result.
workfloworchestration

Orchestration

Conversation-to-PR handoffs

Collaborative agent sessions can turn a Slack or Teams discussion into an issue, code change, or pull request without losing the decisions that shaped the work. The safe pattern is to treat the thread as source material, extract a bounded implementation brief, confirm the repository and branch, preserve the agent's identity on every artifact, and require an independent review before merge.

collaborative-agentshandoffcontextpull-requests

Principles

  • Treat the conversation as evidence, not as an automatically approved specification.
  • Separate who can contribute context from who is authorized to trigger repository changes.
  • Name the target repository and base branch instead of relying on a shared default.
  • Preserve app identity and require an independent human review for agent-authored artifacts.

Steps

  • Summarize the thread into a goal, non-goals, decisions, unknowns, and acceptance criteria.
  • Confirm the repository, base branch, issue, write-authorized owner, and data that may enter agent context.
  • Start one collaborative agent session for one task and keep steering inside its dedicated thread or code channel.
  • Track the generated issue, branch, or pull request as the durable handoff artifact.
  • Review the diff, validation evidence, provenance, and required approvals before merge.

Avoid

  • Sending a long channel history that contains secrets, private data, or unrelated instructions.
  • Assuming every participant who can steer the discussion is allowed to authorize code changes.
  • Letting a channel's default repository or default branch silently choose the change target.
  • Treating a visible collaborative session as a substitute for code review and required checks.