D. J. Bernstein
Data structures and program structures
Rebuilding target files when source files have changed

Target files depend on build scripts

You have a typical Makefile with
     CFLAGS=-O
near the top. You decide to recompile with debugging turned on. You change the line to
     CFLAGS=-g
and type make.

Problem: Nothing happens.

What went wrong?

Answer: make has no idea that the target files depend on the Makefile.

This isn't make's fault. make is perfectly capable of rebuilding target files when the Makefile changes. The author simply has to be honest to make, explaining to make that each target depends on the Makefile. For example, here's part of the Makefile from my ``squeeze'' package, published 1990.02.06 in comp.sources.unix:

     CCOPTS=-O

     squeeze: squeeze.c Makefile
             cc $(CCOPTS) -o squeeze squeeze.c
If Makefile changes, make will rebuild squeeze.

There's a speed problem here. If you change $(CCOPTS), make will also rebuild targets that don't actually depend on $(CCOPTS), such as the package's documentation:

     squeeze.1: squeeze.man Makefile
             nroff $(NROFFOPTS) < squeeze.man > squeeze.1
make has to rebuild squeeze.1 because Makefile has changed. It's as if the user had fired a
     make clean; make
bazooka. This doesn't matter for a tiny package like squeeze, but it's a serious slowdown for big packages.

Some make replacements say ``Switch from make to our zake tool so that you don't have this problem! We remember what commands were invoked by the Zakefile; if the Zakefile changes, we don't rebuild targets whose commands have not actually changed.''

But make is perfectly capable of keeping track of changes in CCOPTS separately from changes in NROFFOPTS. The package author simply has to put the options in two separate files, say cc and nroff:

     squeeze: squeeze.c cc Makefile
             ./cc -o squeeze squeeze.c

     squeeze.1: squeeze.man nroff Makefile
             ./nroff < squeeze.man > squeeze.1
Now a change to the cc file will rebuild squeeze (and other executables) but not squeeze.1, while a change to the nroff file will rebuild squeeze.1 (and other documentation) but not squeeze. Furthermore, cc and nroff can themselves be Makefile targets; this is useful if, for example, the package needs to add the -malign-double compiler option on some machines.

This technique breaks down when your changes to Makefile are changes not just to the build commands but also to the prerequisite lists. For example, if you try to use

     dhcpd: cc dhcpd.deps `cat dhcpd.deps`
             ./cc -o dhcpd `cat dhcpd.deps`
with
     dhcpd.o
     lease.o
in dhcpd.deps then you'll find that make doesn't understand backquotes in prerequisites. Many versions of make allow you to instead say something like
     include dhcpd.make
where dhcpd.make says
     dhcpd: dhcpd.o lease.o dhcpd.make
             ./cc -o dhcpd dhcpd.o lease.o
but they won't update dhcpd properly if dhcpd.make is a target. (Example: Say dhcpd.make is built from dhcpd.c. You change dhcpd.c and type make. First make reads the old dhcpd.make; then it rebuilds dhcpd.make; then it rebuilds dhcpd using the old dhcpd.make commands.)

Exception: GNU make will update dhcpd properly even if dhcpd.make is a target. It does it in a slow way, restarting from scratch after updating dhcpd.make, but this is better than using the wrong commands.

How does redo handle this?

With redo, prerequisite-list generation is a dynamic part of the build process. For example, dhcpd.do can say
     redo-ifchange cc dhcpd.deps
     redo-ifchange `cat dhcpd.deps`
     ./cc -o dhcpd `cat dhcpd.deps`
to indicate that dhcpd depends on cc, dhcpd.deps, and the files listed in dhcpd.deps. You don't have to say redo-ifchange dhcpd.do; that's assumed.

When redo builds dhcpd, it saves these prerequisites in a .redo file. A subsequent redo looks at .redo and can quickly figure out whether dhcpd is up to date. (I'd like to save dhcpd's prerequisites in dhcpd.prereqs rather than in a centralized .redo; this would scale beautifully if a simple prerequisite tracker were added to the UNIX filesystem. On current systems, a single centralized .redo is faster.)

The redo-ifchange cc dhcpd.deps command waits until cc and dhcpd.deps have been rebuilt (if they're out of date), so the redo-ifchange `cat dhcpd.deps` command uses the latest dhcpd.deps. Similarly, if dhcpd.deps turns out to contain dhcpd.o and lease.o, then redo-ifchange `cat dhcpd.deps` waits until dhcpd.o and lease.o have been rebuilt (in parallel; again, that's if they're out of date), so the ./cc -o dhcpd `cat dhcpd.deps` command uses the latest dhcpd.o and lease.o. This top-down approach eliminates any need to start from scratch.