Procedural Macros: Compile-Time Code from Code

Declarative macros are powerful but deliberately limited: they match on syntactic structure and cannot inspect types, enumerate struct fields, or run arbitrary logic. When you write #[derive(Debug)], the compiler needs to examine your struct — how many fields it has, what their types are, whether they are named or positional — and generate a Debug implementation customized to that specific shape. No amount of pattern matching on tokens can do this reliably. Procedural macros can.

A proc macro is a Rust function that takes a TokenStream as input and returns a TokenStream as output. It runs as arbitrary Rust code at compile time. It can parse, inspect, transform, and generate anything. But — and this is the invariant — its output is fed straight to the compiler, where it faces every check hand-written code faces. Type checking. Borrow checking. Lifetime analysis. The macro has full power; the output has zero privilege. This is the token-in, token-out invariant.

Why This Invariant Matters More, Not Less, With More Power

Declarative macros are constrained to syntactic pattern matching. That constraint is itself a safety property — you can trace what they do. Procedural macros remove that constraint. They can run arbitrary code at compile time: network calls, file I/O, complex parsing, anything Rust can do. This makes the output invariant more important, not less.

Consider: a proc macro could generate a thousand lines of code from a single struct definition. Nobody will read those thousand lines. The expansion invariant — that every generated line must pass the full compiler pipeline — is the only thing standing between “powerful metaprogramming” and “untraceable code generation.” The power is real. The safety net is the compiler refusing to lower its standards.

The Wall That macro_rules! Cannot Cross

Here is the concrete problem. You want to auto-generate this:

struct Player {
    name: String,
    score: u32,
    active: bool,
}

impl std::fmt::Debug for Player {
    fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
        f.debug_struct("Player")
            .field("name", &self.name)
            .field("score", &self.score)
            .field("active", &self.active)
            .finish()
    }
}

The implementation depends on knowing the field names and count. A macro_rules! macro sees tokens — it could receive the struct definition, but reliably parsing out fields, types, visibility, generics, and lifetimes through pattern matching alone is impractical. The combinatorial explosion of struct shapes defeats syntax-only matching. This is why #[derive(Debug)] is a procedural macro.

Three Invariant Boundaries

Rust has three kinds of proc macros, each with a different contract about what they can and cannot do to the code they receive:

Derive macros — #[derive(MyTrait)]. They receive the item as read-only input and produce additional code. The invariant: they cannot modify the original item. The struct you wrote stays exactly as written. The macro can only add new impl blocks alongside it.

#[derive(Debug, Clone, MyCustomTrait)]
struct Player {
    name: String,
    score: u32,
}
// Player is unchanged. Derive only added impl blocks.

This constraint is what makes derive macros safe to compose — multiple derives coexist because none can alter the original.

Attribute macros — #[my_attribute]. They receive the item and produce replacement code. The invariant: the original item is consumed. The macro must reproduce anything it wants to keep. Forget to forward visibility or attributes and they vanish silently. More power, more responsibility.

#[log_calls]
fn process_data(input: &str) -> Result<(), Error> {
    // The macro consumes this entire function
    // and returns a transformed version
}

Function-like macros — my_macro!(...). Arbitrary tokens in, arbitrary tokens out. The invariant: the output must be valid Rust. No structural constraint on what the macro does internally, but the compiler accepts nothing it has not verified.

let query = sql!("SELECT * FROM users WHERE age > 21");
// The macro can validate SQL at compile time
// The output is still compiler-checked Rust

All three share one architectural constraint: proc macros must live in their own crate, marked with proc-macro = true in Cargo.toml. This enforces a crate boundary between compile-time code and runtime code — the macro crate is compiled before the crates that use it.

The Ecosystem: Parse, Transform, Generate

Three crates make proc macros practical:

• syn — parses a TokenStream into a structured AST (the struct’s fields, a function’s signature, an enum’s variants)
• quote — turns Rust-like quasi-quoted syntax back into a TokenStream
• proc-macro2 — a bridge type that works in non-macro contexts (for testing)

The pattern is: parse with syn, transform the data, generate with quote. Every step operates on structured data, not raw text.

A Derive Macro: The Invariant End to End

Let’s build #[derive(Describe)] — a macro that generates a describe() method listing a struct’s fields. This shows every invariant in action.

The proc macro crate (describe_macro/Cargo.toml):

[lib]
proc-macro = true

[dependencies]
syn = { version = "2", features = ["full"] }
quote = "1"
proc-macro2 = "1"

The macro itself — a function from TokenStream to TokenStream:

// describe_macro/src/lib.rs
use proc_macro::TokenStream;
use quote::quote;
use syn::{parse_macro_input, DeriveInput, Data, Fields};

#[proc_macro_derive(Describe)]
pub fn derive_describe(input: TokenStream) -> TokenStream {
    let input = parse_macro_input!(input as DeriveInput);
    let name = &input.ident;

    let field_descriptions = match &input.data {
        Data::Struct(data_struct) => match &data_struct.fields {
            Fields::Named(fields) => {
                let descs: Vec<_> = fields.named.iter().map(|f| {
                    let fname = f.ident.as_ref().unwrap();
                    let ftype = &f.ty;
                    let fname_str = fname.to_string();
                    let ftype_str = quote!(#ftype).to_string();
                    quote! {
                        format!("  {}: {}", #fname_str, #ftype_str)
                    }
                }).collect();
                quote! { vec![#(#descs),*] }
            }
            _ => quote! { vec![String::from("  (non-named fields)")] },
        },
        _ => quote! { vec![String::from("  (not a struct)")] },
    };

    let name_str = name.to_string();
    let expanded = quote! {
        impl #name {
            pub fn describe() -> String {
                let fields = #field_descriptions;
                format!("{} {{\n{}\n}}", #name_str, fields.join("\n"))
            }
        }
    };

    TokenStream::from(expanded)
}

What happens, traced through the invariants:

1. parse_macro_input! converts raw tokens into DeriveInput — a structured AST with the struct name, fields, types, and attributes. This is what macro_rules! cannot do: semantic inspection of the item.
2. The macro enumerates fields by name and type, building a format string for each.
3. quote! { ... } generates an impl block as tokens. #name and #field_descriptions are interpolations.
4. The generated impl block is returned as a TokenStream — and the compiler checks it like hand-written code. Type error? Caught. Ownership violation? Caught.

Using the macro:

use describe_macro::Describe;

#[derive(Describe)]
struct Config {
    host: String,
    port: u16,
    debug: bool,
}

fn main() {
    println!("{}", Config::describe());
}

Config {
  host: String
  port: u16
  debug: bool
}

The macro inspected three fields at compile time and generated a method. The struct is unchanged — the derive invariant (cannot modify the original) held. The generated impl block was type-checked — the expansion invariant held.

Attribute Macros: Consuming and Replacing Under Contract

An attribute macro wraps or transforms a function. Here is #[log_calls]:

use proc_macro::TokenStream;
use quote::quote;
use syn::{parse_macro_input, ItemFn};

#[proc_macro_attribute]
pub fn log_calls(_attr: TokenStream, item: TokenStream) -> TokenStream {
    let input_fn = parse_macro_input!(item as ItemFn);
    let fn_name = &input_fn.sig.ident;
    let fn_name_str = fn_name.to_string();
    let fn_block = &input_fn.block;
    let fn_sig = &input_fn.sig;
    let fn_vis = &input_fn.vis;

    let expanded = quote! {
        #fn_vis #fn_sig {
            let _start = std::time::Instant::now();
            println!("[ENTER] {}", #fn_name_str);
            let _result = (|| #fn_block)();
            println!("[EXIT]  {} ({:?})", #fn_name_str, _start.elapsed());
            _result
        }
    };

    TokenStream::from(expanded)
}

#[log_calls]
fn compute(x: i32) -> i32 {
    x * x + 1
}

fn main() {
    let result = compute(5);
    println!("result: {}", result);
}

[ENTER] compute
[EXIT]  compute (245ns)
result: 26

The original function was consumed. The macro reproduced its visibility (#fn_vis), signature (#fn_sig), and wrapped its body with timing. The contract: if the macro forgets to forward something — an #[inline] attribute, a pub visibility — it disappears. The compiler does not warn. The macro author bears the responsibility that comes with the power to replace.

Function-Like Macros: Compile-Time Validation

The most compelling use: catching errors before runtime.

use proc_macro::TokenStream;

#[proc_macro]
pub fn make_answer(_input: TokenStream) -> TokenStream {
    "fn answer() -> i32 { 42 }".parse().unwrap()
}

make_answer!();

fn main() {
    println!("{}", answer()); // 42
}

A hypothetical sql! macro could parse SQL at compile time and reject invalid queries before the program ever runs. The token-in, token-out invariant makes this safe: the macro runs arbitrary validation logic, but its output is still Rust tokens that the compiler will fully verify.

Error Reporting: The Macro’s Diagnostic Contract

When a proc macro encounters invalid input, it should produce a clear compile error — not panic:

use syn::Error;

#[proc_macro_derive(Describe)]
pub fn derive_describe(input: TokenStream) -> TokenStream {
    let input = parse_macro_input!(input as DeriveInput);

    match &input.data {
        Data::Struct(_) => { /* generate code */ }
        _ => {
            return Error::new_spanned(
                &input.ident,
                "Describe can only be derived for structs"
            ).to_compile_error().into();
        }
    }
    // ...
}

#[derive(Describe)]
enum Color { Red, Green, Blue }

error: Describe can only be derived for structs
 --> src/main.rs:2:6
  |
2 | enum Color { Red, Green, Blue }
  |      ^^^^^

The error points to the exact offending token. Good proc macros honor the same diagnostic standard the compiler itself sets — because from the user’s perspective, a macro error IS a compiler error.

The Token-In, Token-Out Invariant, Summarized

Macro Kind	Input	Output	Contract
Derive	Item definition (read-only)	Additional impl blocks	Cannot modify original item
Attribute	Item definition (consumed)	Replacement item	Must reproduce what it wants to keep
Function-like	Arbitrary tokens	Arbitrary tokens	Output must be valid Rust
All three	TokenStream	TokenStream	Output is type-checked, borrow-checked, lifetime-checked

---

Procedural macros remove the syntactic constraints of macro_rules! — they can inspect struct fields, generate trait implementations, validate domain-specific languages, run arbitrary logic at compile time. But removing one constraint does not remove the others. The output is still tokens. The compiler still checks those tokens with full rigor. A proc macro can generate anything; the compiler will accept nothing it has not proved correct. Power to generate. Zero privilege in the output. That is the contract. Next up: macro hygiene and design — the invariant that prevents macro-generated names from colliding with your code, and the principles for knowing when not to write a macro at all.