Loose Coupling/High Cohesion

Group your packages by types and the functions that manipulate those types. More functional organization means looser coupling and high cohesion.

This is the project directory layout I have adopted for all my projects. It is intuitive, semantically rich and most importantly it is loosely coupled and highly coherent.

Context

This is a project that is deployed to Google Cloud Platform.

Specifically Google App Engine. But this project layout works equally well for command line only tools and stand along desktop apps so far.

Each package is a specific domain or feature. Do not get caught up on the the term "domain", there is some "domain driven design" ideas but none of the dogma associated with it.

Specifically having a models package that has all the models in it. I started trying to do that and it was just a tightly coupled incoherent mess to navigate around, even in the awesome Goland IDE from Jetbrains.

So I inverted that idea and put all the code supporting a specific type(s) together like you would do in most functional languages and this worked very well.

The filenames are semantically descriptive and should be self explanatory to anyone that checks out the code base for the first time. They should be able to intuitively find what they are interested in very quickly without a lot of searching.

For a particular domain/feature/functionality package the files that are most common are going to be. They are detailed after the example tree.

├── account
│   ├── functions.go
│   ├── handlers.go
│   ├── tasks.go
│   └── types.go
├── cuid2
│   └── cuid2.go
├── gcp
│   ├── auth
│   │   ├── functions.go
│   │   ├── handlers.go
│   │   ├── middleware.go
│   │   ├── tasks.go
│   │   └── types.go
│   └── secrets
│   ├── functions.go
│   └── types.go
├── go.mod
├── go.sum
├── main.go
├── server
│   ├── functions.go
│   ├── middleware.go
│   ├── must_functions.go
│   └── types.go
├── timestamp
│   ├── functions.go
│   ├── functions_test.go
│   ├── types.go
│   └── types_test.go
└── external_service
 ├── errors.go
 ├── datastore.go
 ├── functions.go
 ├── handlers.go
 ├── secrets.go
 ├── service.go
 ├── tasks.go
 └── types.go

Files that will be in most every package:

types.go

This file contains all the types for the package, both exported and non-exported.

For most packages both interfaces and structs and any struct receiver functions are all in the types.go file.

In rare cases the types.go file may need to be split up into an interfaces.go and a structs.go just because the number of interfaces and structs ends up becoming cumbersome. This should make you take pause of your package and see if it is overly generalized, or the types are too granular, or that you are just doing it wrong.

I have a well rounded background in functional and imperative programming languages and I find that I use receiver functions very sparingly in Go. If your types look like Java classes then you are doing it wrong.

My types in Go usually only have interfaces that make sense on the type, like Stringer or Marshal/UnmarshalJson or Marshal/UnmarshalBinary and Marshal/UnmarshalText

Instead of putting a AsTime() time.Time receiver function on my custom Timestamp type, I would rather have it a stand along func AsTime(ts Timestamp) time.Time function. It makes my Timestamp type loosely coupled to the time.Time type and still maintains high cohesion/locality. It also limits interface explosion. You can also pass the function into other functions if you need a different implementation for some reason as long as the signature is func(ts Timestamp) time.Time.

functions.go

This file contains all the functions for the package, both exported and non-exported.

These are functions that are actually functions, not receiver functions, those should be in the same file as the struct they are on for high cohesion.

Exceptions:

There are always exceptions!

For the cuid2 package there is only one type and a handful of functions, so I just put them all in a single file with the same name of the package. This indicates that everything is self contained in this single file. If that file should grow to be more complicated or unwieldy from size it would then be refactored into types.go and functions.go at the minimum.

This is in service to the high cohesion principle. If everything can easily be in a single file, then it should be.

You will also notice there are no packages or files with names like util or common.go or helper, those names might as well just be djjsdfiyhfye, they are actually worse; they do not convey anymore more useful information than line noise, but since they are words you infer what they mean without actually knowing what the intent was. They are misinformation at best, where as random characters would at least tell you immediately that those files were suspect and obfuscations at best.

Web Applications / APIs

server or app

The server or app package is a horizontal cross cutting package that should not import any of the application specific packages but be imported by many of them to access high level server functionality. This can be `apps for command line or desktop applications.

must_functions.go

This file is functions that must not fail, and will stop the server if they do. In most cases these are functions that are either critical to the server and errors are unrecoverable, or they are wrappers around things in functions.go that are processing known good data that if an error does occur it is because of major data corruption and should stop the server.

Things like must_parse_time(s string) time.Time where you know the format of the string is valid because it came from a know source or has been previously validated in the code path.

handlers.go

This is where exported API handler functions would be found, and any non-exported support functions that might be specific to these handlers.

middleware.go

This is handlers that are middleware that support the package they are in. Server wide middleware would be in the server package. authentication middleware would be in the authentication package. You get the idea, loosely coupled and highly coherent.

service.go

This particular project access the YouTube Data API V3 so there is a youtube package. The service.go file contains all the functions to actually make the calls to the youtube API. These could be stand alone functions or a struct with receiver functions that implemented the calls so they could share construction and pooling of an API specific client object and its creation.

secrets.go

This file contains package specific functions and types to access the Google Cloud Secrets Manager API calls that are in server/secrets.go. That file contains general abstractions over the secret manager client and calls, like creating a secret, adding a version and enabling it and disabling/deleted previous versions all in one transaction.

datastore.go

This project stores data in the Google Cloud Datastore. This could be called repository.go, and it was, I decided that was to generalized and did not convey enough semantics. datastore.go tells you it is a datastore implementation in the context that this is a Google Cloud Platform application.

If I was going to support multiple storage implementations I would put that interface into repository.go and the specific stores in datastore.go, postgres.go, redis.go, etc.

tasks.go

This file contains the functions to create Google Cloud Tasks submissions. If they take specific structs that are only used in the task functions to create the tasks, then I make then non-exported and put them in the tasks.go file, if they are used elsewhere I put them in types.go. Cohesion over rote consistency.

errors.go

I am experimenting with custom error structs in the on package in this project, that is what this file contains, the error types and the functions to create new instances.

I have toyed with making this a sub-package, but since Go does not support sub-package namespaces (pet peeve of mine) and flatter packages are idiomatic Go, I am doing it this way for right now. This is an experiment, it may change depending on how successful it is.

gcp/auth gcp/secrets

I am still debating these nested packages. This is a Google Cloud Project application and these packages contain Google Cloud Project specific authentication and secrets manager code.

I initially put all the code just under gcp package, but mixing the authentication functions and types with the secrets manager functions and types go messy very quickly. Tightly coupling the disparate features to each other in location if nothing else. And weakening cohesion at the same time, because it was hard to see what was related and unrelated with both features mixed in the same types.go and functions.go files.

I am satisfied with this right now, but still not completely. Putting an auth and secret package in the root of the project would lose the information that the outer gcp package conveys semantically, so it stays until I have a better solution. There may not be one.

There is no one correct project layout:

There are ones that are more correct and this is the most semantically rich, intuitive and idiomatic one I have seen anyone come up with so far.

• It is adaptable but consistent and predictable.

• It is simple but scales out to more complex cases when needed.

• It is minimal without being terse and expressive without being verbose.

• It is opinionated without being dogmatic.