What are the file limits in Git number and size

Navigating version control systems effectively requires a deep understanding of their inherent capabilities and limitations. One common question that often arises, especially for developers working with diverse asset types, is: what are the file limits in Git (number and size)? While Git is renowned for its efficiency with text-based code, its distributed architecture and snapshot-based approach introduce nuanced considerations when dealing with extremely large individual files or repositories containing millions of objects. Unlike some centralized systems with explicit hard limits, Git’s “limits” are more practical and performance-driven, impacting everything from cloning speed to disk space consumption. Understanding these factors is crucial for maintaining a smooth development workflow and preventing bottlenecks as your projects scale.

Understanding Git’s Design Philosophy and its Impact on File Limits

Git was fundamentally designed for tracking changes in source code, which primarily consists of small, text-based files. Its core mechanism involves storing snapshots of your project’s entire file system at each commit, rather than just the differences (deltas) between versions. This design choice makes operations like branching and merging incredibly fast, but it presents challenges when confronting large binary files like images, audio, or video. When a large file changes even slightly, Git stores a new full copy of that file in the repository history, quickly inflating the repository size.

This approach means that while there isn’t a strict, hard-coded maximum file size enforced by Git itself, the practical implications of storing massive files directly in the repository can be severe. Each commit creates a new snapshot, and if those snapshots contain multi-gigabyte files, your repository can swell to unmanageable proportions. This significantly slows down common Git operations such as cloning, fetching, and pushing, as every developer needs to download the entire history, including all versions of those large files. For instance, a 1GB video file updated weekly could easily add tens of gigabytes to your repository within months.

Furthermore, Git’s indexing system, which helps track millions of files, is optimized for speed with typical codebases. However, when faced with an extremely high number of files (e.g., millions of tiny configuration files or assets), the index itself can become very large. This can lead to increased memory usage and slower performance for commands like git status or git ls-files, even if individual files are small. The key takeaway is that Git prioritizes efficiency for its intended use case, and deviating too far from that can expose practical performance ceilings rather than explicit architectural limits.

Practical Limitations on File Size and Performance

While Git doesn’t have a hard ceiling on individual file size, files exceeding 100MB are generally considered “large” and can begin to degrade performance significantly. Files in the gigabyte range will almost certainly cause issues. When you clone a repository, Git must download every version of every file. If your history includes multiple versions of multi-gigabyte files, the initial clone can take hours or even fail due to network timeouts or disk space exhaustion. This directly impacts developer onboarding and CI/CD pipeline efficiency.

The impact extends beyond initial cloning. Operations like committing, pushing, and pulling also suffer. Git needs to process these large objects, pack them, and transmit them across the network. This can consume substantial CPU resources and bandwidth. For teams collaborating on projects involving design assets, scientific data, or game development, this becomes a critical bottleneck. The common advice from the Git community is to avoid committing large binary files directly into your repository due to these performance implications and the fact that Git’s diffing algorithm is ineffective for binaries.

For scenarios involving large files, Git Large File Storage (Git LFS) is the industry-standard solution. Instead of embedding large files directly in the Git repository, Git LFS replaces them with small “pointer” files. The actual large file content is stored on a separate LFS server. This means your Git repository remains lightweight, and developers only download the specific versions of large files they need for their current branch, significantly improving performance. For more technical details on how Git LFS operates, you can refer to its official documentation on Git LFS.

Managing a High Number of Files in Git Repositories

Beyond individual file size, the sheer number of files within a repository can also pose practical challenges. While Git is engineered to handle substantial projects, a repository containing hundreds of thousands or even millions of files can lead to performance degradation. The Git index, a crucial component that tracks all files in your working directory and staging area, can grow considerably with a high file count. A large index means that commands interacting with the index, such as git status, git add, and git commit, will take longer to execute.

This phenomenon is particularly noticeable in large monorepos where many independent components or services reside within a single Git repository. Even if individual files are small, the aggregate effect of millions of files can slow down development workflows. For example, some large tech companies manage monorepos with millions of files and hundreds of thousands of directories. While Git can technically track this many files, specialized tooling and careful configurations, such as utilizing sparse-checkout or shallow clones, become essential to maintain developer productivity. Without these optimizations, operations that iterate over the entire working tree can become prohibitively slow.

When working with repositories that contain an exceptionally high number of files, it’s vital to implement strategies that minimize the local working set for developers. This includes leveraging Git’s advanced features to only check out the necessary subdirectories or to limit the depth of the repository history. These techniques significantly reduce the amount of data Git needs to manage locally, thereby mitigating the performance impact of a vast file count. Ignoring irrelevant files using a well-structured .gitignore file is also a foundational best practice, preventing unnecessary files from ever being added to the index and repository history.

Effectively managing repositories with large files or a high file count requires proactive strategies. The primary solution for large binary files is Git Large File Storage (LFS). LFS works by replacing large files in your repository with text pointers Question & Answer :

Does anyone know what are the Git limits for number of files and size of files?

This message from Linus himself can help you with some other limits

[…] CVS, ie it really ends up being pretty much oriented to a “one file at a time” model.

Which is nice in that you can have a million files, and then only check out a few of them - you’ll never even see the impact of the other 999,995 files.

Git fundamentally never really looks at less than the whole repo. Even if you limit things a bit (ie check out just a portion, or have the history go back just a bit), git ends up still always caring about the whole thing, and carrying the knowledge around.

So git scales really badly if you force it to look at everything as one huge repository. I don’t think that part is really fixable, although we can probably improve on it.

And yes, then there’s the “big file” issues. I really don’t know what to do about huge files. We suck at them, I know.

See more in my other answer: the limit with Git is that each repository must represent a “coherent set of files”, the “all system” in itself (you can not tag “part of a repository”).
If your system is made of autonomous (but inter-dependent) parts, you must use submodules.

As illustrated by Talljoe’s answer, the limit can be a system one (large number of files), but if you do understand the nature of Git (about data coherency represented by its SHA-1 keys), you will realize the true “limit” is a usage one: i.e, you should not try to store everything in a Git repository, unless you are prepared to always get or tag everything back. For some large projects, it would make no sense.

For a more in-depth look at git limits, see “git with large files”
(which mentions git-lfs: a solution to store large files outside the git repo. GitHub, April 2015)

The three issues that limits a git repo:

• huge files (the xdelta for packfile is in memory only, which isn’t good with large files)
• huge number of files, which means, one file per blob, and slow git gc to generate one packfile at a time.
• huge packfiles, with a packfile index inefficient to retrieve data from the (huge) packfile.

A more recent thread (Feb. 2015) illustrates the limiting factors for a Git repo:

Will a few simultaneous clones from the central server also slow down other concurrent operations for other users?

There are no locks in server when cloning, so in theory cloning does not affect other operations. Cloning can use lots of memory though (and a lot of cpu unless you turn on reachability bitmap feature, which you should).

Will ‘git pull’ be slow?

If we exclude the server side, the size of your tree is the main factor, but your 25k files should be fine (linux has 48k files).

‘git push’?

This one is not affected by how deep your repo’s history is, or how wide your tree is, so should be quick..

Ah the number of refs may affect both git-push and git-pull.
I think Stefan knows better than I in this area.

‘git commit’? (It is listed as slow in reference 3.) ‘git status’? (Slow again in reference 3 though I don’t see it.)
(also git-add)

Again, the size of your tree. At your repo’s size, I don’t think you need to worry about it.

Some operations might not seem to be day-to-day but if they are called frequently by the web front-end to GitLab/Stash/GitHub etc then they can become bottlenecks. (e.g. ‘git branch --contains’ seems terribly adversely affected by large numbers of branches.)

git-blame could be slow when a file is modified a lot.