LehnerBuild

Javascript

node and Error EMFILE too many open files

25 September 2026 · 8 min read

node and Error EMFILE too many open files

Node.js developers frequently encounter the dreaded “Error: EMFILE, too many open files” message. This frustrating error typically arises when a Node.js application attempts to open more files than the operating system allows. Understanding the underlying causes and implementing effective solutions is crucial for maintaining a smooth and efficient application. This article delves into the intricacies of the EMFILE error, exploring its origins, providing practical solutions, and offering preventative measures to avoid future occurrences. We’ll cover everything from adjusting system limits to optimizing your code for better file handling.

Understanding the EMFILE Error

The “EMFILE” error signifies that your Node.js process has hit the limit on the number of files it can simultaneously open. Every time your application interacts with a file—reading, writing, or even executing—it requires a file descriptor. These descriptors are limited resources managed by the operating system. When your application requests more descriptors than available, the OS throws the EMFILE error. This limit isn’t fixed; it varies depending on the OS and its configuration. In essence, it’s a resource bottleneck.

Imagine a librarian trying to manage too many borrowers simultaneously. Each borrower represents a file, and the librarian can only handle a certain number of requests at once. When the limit is reached, new borrowers are turned away, similar to how the OS rejects further file access attempts, resulting in the EMFILE error. Understanding this analogy helps visualize the resource constraint at play.

Diagnosing the Root Cause

Pinpointing the source of the EMFILE error often involves investigating how your application handles files. Are you frequently opening files without properly closing them? Are you using libraries or modules that might be holding onto file descriptors longer than necessary? These are common culprits. Analyzing your code’s file I/O operations is the first step towards resolving the issue.

Begin by scrutinizing functions that interact with the file system. Look for potential areas where files might be opened and not closed correctly, especially within loops or asynchronous operations. Unclosed files consume valuable file descriptors, leading to the EMFILE error. Tools like lsof (list open files) on Linux/macOS can provide a snapshot of currently open files by your process, aiding in identifying the source of the problem.

Effective Solutions for EMFILE Errors

Several strategies can mitigate EMFILE errors. One common approach is increasing the system’s open file limit. On Linux and macOS, this involves modifying the ulimit setting. However, this is a temporary fix and doesn’t address the underlying code inefficiencies. A more sustainable solution involves optimizing your code to handle files more efficiently.

  1. Increase the ulimit (Unix-like systems): Use the command ulimit -n [new_limit] to raise the limit. Be mindful of system-wide resource constraints. This modification might require administrator privileges.
  2. Properly Close Files: Ensure that every fs.open() or fs.createReadStream() call is paired with a corresponding fs.close() or an appropriate stream ’end’ event handler. This is crucial for freeing up file descriptors.
  3. Utilize Streams: Instead of reading entire files into memory, leverage Node.js streams to process data in chunks. This reduces memory footprint and minimizes the number of open file descriptors needed at any given time.

Consider implementing graceful error handling to catch EMFILE errors and retry operations after a short delay. This can improve application resilience. Remember, treating the symptom alone isn’t enough; addressing the root cause—inefficient file handling—is essential for long-term stability.

Preventative Measures and Best Practices

Preventing EMFILE errors involves adopting proactive coding practices that prioritize efficient file management. Using libraries that manage file descriptors effectively can significantly reduce the risk of encountering this issue. Furthermore, incorporating regular code reviews can help identify and address potential file handling bottlenecks before they escalate into production errors.

One valuable approach is to use connection pooling for database interactions. This technique manages a pool of database connections, reusing them as needed, instead of constantly opening and closing connections, which can contribute to EMFILE errors. This is especially pertinent when dealing with frequent database queries. By reusing connections, you minimize the number of simultaneously open files, reducing the strain on system resources and preventing potential EMFILE issues.

  • Employ file descriptor pooling libraries.
  • Conduct regular code reviews to identify potential file handling issues.

Check out this helpful resource: Node.js File System Documentation

Infographic Placeholder: (Visual representation of file descriptors, OS limits, and best practices)

Incorporating these practices into your development workflow can significantly reduce the likelihood of encountering EMFILE errors, leading to a more robust and reliable Node.js application. See our article on Troubleshooting Common Node.js Errors for more debugging tips.

Frequently Asked Questions

Q: Why does the EMFILE error occur more often in Node.js than other environments?

A: Node.js’s asynchronous nature often involves handling numerous concurrent operations, which can lead to a higher demand for file descriptors compared to synchronous environments.

By understanding the nuances of file descriptors, implementing efficient file handling techniques, and adopting preventative measures, developers can effectively address and avoid EMFILE errors, ensuring the smooth operation of their Node.js applications. For further assistance, consider exploring resources like the official Node.js documentation and community forums dedicated to troubleshooting similar issues. Remember, proper file management is key to a stable and performant application. Start optimizing your code today to prevent future EMFILE headaches and ensure your application runs smoothly.

Explore further related topics like asynchronous programming, stream management, and efficient resource utilization in Node.js to strengthen your understanding and improve your coding practices. Dive deeper into the world of server-side JavaScript and master the art of building robust and scalable applications. W3Schools Node.js Tutorial and Official Node.js Documentation are excellent starting points. Also, check out this informative resource on Stack Overflow - EMFILE Issues for community-driven solutions and discussions.

Question & Answer :
For some days I have searched for a working solution to an error

Error: EMFILE, too many open files

It seems that many people have the same problem. The usual answer involves increasing the number of file descriptors. So, I’ve tried this:

sysctl -w kern.maxfiles=20480

The default value is 10240. This is a little strange in my eyes, because the number of files I’m handling in the directory is under 10240. Even stranger, I still receive the same error after I’ve increased the number of file descriptors.

Second question:

After a number of searches I found a work around for the “too many open files” problem:

var requestBatches = {}; function batchingReadFile(filename, callback) { // First check to see if there is already a batch if (requestBatches.hasOwnProperty(filename)) { requestBatches[filename].push(callback); return; } // Otherwise start a new one and make a real request var batch = requestBatches[filename] = [callback]; FS.readFile(filename, onRealRead); // Flush out the batch on complete function onRealRead() { delete requestBatches[filename]; for (var i = 0, l = batch.length; i < l; i++) { batch[i].apply(null, arguments); } } } function printFile(file){ console.log(file); } dir = "/Users/xaver/Downloads/xaver/xxx/xxx/" var files = fs.readdirSync(dir); for (i in files){ filename = dir + files[i]; console.log(filename); batchingReadFile(filename, printFile);

Unfortunately I still recieve the same error. What is wrong with this code?

For when graceful-fs doesn’t work… or you just want to understand where the leak is coming from. Follow this process.

(e.g. graceful-fs isn’t gonna fix your wagon if your issue is with sockets.)

From My Blog Article: http://www.blakerobertson.com/devlog/2014/1/11/how-to-determine-whats-causing-error-connect-emfile-nodejs.html

How To Isolate

This command will output the number of open handles for nodejs processes:

lsof -i -n -P | grep nodejs 

COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME ... nodejs 12211 root 1012u IPv4 151317015 0t0 TCP 10.101.42.209:40371->54.236.3.170:80 (ESTABLISHED) nodejs 12211 root 1013u IPv4 151279902 0t0 TCP 10.101.42.209:43656->54.236.3.172:80 (ESTABLISHED) nodejs 12211 root 1014u IPv4 151317016 0t0 TCP 10.101.42.209:34450->54.236.3.168:80 (ESTABLISHED) nodejs 12211 root 1015u IPv4 151289728 0t0 TCP 10.101.42.209:52691->54.236.3.173:80 (ESTABLISHED) nodejs 12211 root 1016u IPv4 151305607 0t0 TCP 10.101.42.209:47707->54.236.3.172:80 (ESTABLISHED) nodejs 12211 root 1017u IPv4 151289730 0t0 TCP 10.101.42.209:45423->54.236.3.171:80 (ESTABLISHED) nodejs 12211 root 1018u IPv4 151289731 0t0 TCP 10.101.42.209:36090->54.236.3.170:80 (ESTABLISHED) nodejs 12211 root 1019u IPv4 151314874 0t0 TCP 10.101.42.209:49176->54.236.3.172:80 (ESTABLISHED) nodejs 12211 root 1020u IPv4 151289768 0t0 TCP 10.101.42.209:45427->54.236.3.171:80 (ESTABLISHED) nodejs 12211 root 1021u IPv4 151289769 0t0 TCP 10.101.42.209:36094->54.236.3.170:80 (ESTABLISHED) nodejs 12211 root 1022u IPv4 151279903 0t0 TCP 10.101.42.209:43836->54.236.3.171:80 (ESTABLISHED) nodejs 12211 root 1023u IPv4 151281403 0t0 TCP 10.101.42.209:43930->54.236.3.172:80 (ESTABLISHED) ....

Notice the: 1023u (last line) - that’s the 1024th file handle which is the default maximum.

Now, Look at the last column. That indicates which resource is open. You’ll probably see a number of lines all with the same resource name. Hopefully, that now tells you where to look in your code for the leak.

If you don’t know multiple node processes, first lookup which process has pid 12211. That’ll tell you the process.

In my case above, I noticed that there were a bunch of very similar IP Addresses. They were all 54.236.3.### By doing ip address lookups, was able to determine in my case it was pubnub related.

Command Reference

Use this syntax to determine how many open handles a process has open…

To get a count of open files for a certain pid

I used this command to test the number of files that were opened after doing various events in my app.

lsof -i -n -P | grep "8465" | wc -l 

# lsof -i -n -P | grep "nodejs.*8465" | wc -l 28 # lsof -i -n -P | grep "nodejs.*8465" | wc -l 31 # lsof -i -n -P | grep "nodejs.*8465" | wc -l 34

What is your process limit?

ulimit -a

The line you want will look like this:

open files (-n) 1024

Permanently change the limit:

  • tested on Ubuntu 14.04, nodejs v. 7.9

In case you are expecting to open many connections (websockets is a good example), you can permanently increase the limit:

  • file: /etc/pam.d/common-session (add to the end)

    session required pam_limits.so

  • file: /etc/security/limits.conf (add to the end, or edit if already exists)

    root soft nofile 40000 root hard nofile 100000

  • restart your nodejs and logout/login from ssh.

  • this may not work for older NodeJS you’ll need to restart server

  • use instead of if your node runs with different uid.