Chapter 4: Modules Should Be Deep

Core Idea

The best modules have simple interfaces that hide powerful functionality. A module’s interface is its cost to the rest of the system and its functionality is its benefit, so make modules deep, not small.

Frameworks Introduced

• Interface vs. implementation: think of every module in two parts.
  • The interface is “everything that a developer working in a different module must know in order to use the given module.” It says what the module does, not how.
  • The implementation is the code that keeps the interface’s promises.
  • A developer working in a module must understand its interface and implementation, plus the interfaces of the modules it calls, but never the implementations of other modules.
• Formal vs. informal interface:
  • Formal: what the code states and the language can check, such as method signatures, public methods, and public variables.
  • Informal: high-level behavior (for example, that a function deletes the file named by an argument) and usage constraints (for example, one method must be called before another). Only comments can describe these. “For most interfaces the informal aspects are larger and more complex than the formal aspects.”
  • Rule: “if a developer needs to know a particular piece of information in order to use a module, then that information is part of the module’s interface.”
• Abstraction: “a simplified view of an entity, which omits unimportant details.” It can fail in two ways:
  1. It includes details that are not important, which adds cognitive load.
  2. It omits details that are important, which creates obscurity. This is a false abstraction: it looks simple but isn’t.
  • How: understand what is important, and look for designs that minimize how much information is important.
• Deep vs. shallow modules (Figure 4.1): draw each module as a rectangle. Its area is the functionality (the benefit; more is better). Its top edge is the interface (the cost; shorter is better). A deep module is a tall, narrow rectangle with a short top edge. A shallow module is a wide, flat rectangle with a long top edge over little area.
  • When to use: comparing designs for any class, method, subsystem, or service.
  • How: prefer the design with the most functionality behind the simplest interface. “Interfaces are good, but more, or larger, interfaces are not necessarily better!”
• Make the common case simple: if most users need only a few features of an interface, its effective complexity is just the complexity of those features. Put rare features where common users never see them.

Key Concepts

• Module: any unit of code with an interface and an implementation: a class, a method, a function, a subsystem, or a service (whose interface may be kernel calls or HTTP requests).
• Deep module: lots of functionality hidden behind a simple interface. Examples: Unix file I/O, a garbage collector, a balanced-tree class.
• Shallow module: an interface that is complicated relative to its functionality. Example: a linked-list class.
• Classitis: the mistaken view that “classes are good, so more classes are better,” which produces many small shallow classes whose interfaces add up to great system complexity, plus verbose boilerplate.
• Red Flag: Shallow Module: the benefit of not knowing how it works is cancelled by the cost of learning its interface. “Small modules tend to be shallow.”

Mental Models

• Think of depth as cost versus benefit. The best modules give the greatest benefit at the least cost.
• Think of a garbage collector as the deepest module possible: it has no interface at all. Adding it shrinks the system’s interface, since nobody has to free objects.
• Use everyday abstractions as a check. A microwave hides power electronics behind a few buttons, and a car hides motors and anti-lock brakes behind a wheel and pedals.
• Expose an implementation detail only if users need it to work correctly. A file system can hide block allocation, but it must expose its rules for flushing data to storage, because databases depend on them.
• Prefer English over formal specifications for the informal interface. The author thinks English is more intuitive for developers.

Anti-patterns

• “Classes should be small” / “Any method longer than N lines should be divided” (with N sometimes as low as 10): this produces many shallow classes and methods.
• A method that only wraps one statement: the student example addNullValueForAttribute(String attribute) just calls data.put(attribute, null). It offers no abstraction, its documentation would be longer than its code, and it takes more keystrokes to call than to write the statement.
• Making users ask for the feature everyone needs: Java requires a separate BufferedInputStream for buffering, and forgetting it makes I/O slow.

Code Examples

The Unix file I/O interface, five calls whose signatures have not changed while their implementation grew to hundreds of thousands of lines:

int open(const char* path, int flags, mode_t permissions);
ssize_t read(int fd, void* buffer, size_t count);
ssize_t write(int fd, const void* buffer, size_t count);
off_t lseek(int fd, off_t offset, int referencePosition);
int close(int fd);

• What it demonstrates: a deep interface. The implementation handles on-disk layout, directories and path lookup, permissions, interrupt handling, scheduling of concurrent access, caching, and multiple device types, and none of it is visible to callers. Sequential access is the default, and lseek exists for the rarer random access.

Worked Example

Java I/O versus Unix I/O. To read serialized objects from a file in Java, you create three objects:

FileInputStream fileStream =
    new FileInputStream(fileName);
BufferedInputStream bufferedStream =
    new BufferedInputStream(fileStream);
ObjectInputStream objectStream =
    new ObjectInputStream(bufferedStream);

1. FileInputStream does only basic I/O. BufferedInputStream adds buffering, and ObjectInputStream adds serialization. The first two objects are never used again after the file is opened.
2. The likely defense is that not everyone wants buffering, so it should be optional. The author agrees that choice is good, but says the common case should be simple: almost every user wants buffering, so it should be the default.
3. The fix: provide buffering automatically, and offer a separate way to disable or replace it, such as a different constructor or a method. Most developers then never need to know the option exists.
4. Unix did this right. Sequential I/O is the default, and random access needs lseek, which sequential users never have to learn.

Key Takeaways

1. Judge a module by the ratio of the functionality it hides to the complexity of its interface.
2. Count informal behavior and usage constraints as part of the interface, and document them.
3. Omit every unimportant detail from an abstraction, but never omit an important one.
4. Resist classitis and rules about maximum method length; many small modules add interface complexity.
5. Design interfaces so the common case is the simplest one, and make rare options invisible to most users.

Connects To

• Ch 2: an interface is a dependency; the C = Σ c_p t_p formula explains why the common case matters most.
• Ch 5: information hiding is the main technique for making modules deep.
• Ch 6: somewhat general-purpose interfaces tend to be deeper.
• Ch 7: pass-through methods and decorators are common shallow designs.
• Ch 9: when splitting methods or classes makes things shallow, and when it doesn’t.
• Ch 13: interface comments describe the informal parts of an interface.