Python List all Files in a Directory

There are several ways to list all the files in a directory in Python. Each method has its own advantages and disadvantages. In this article, we will explore some of the most commonly used methods, including os.listdir(), os.scandir(), os.walk(), the glob module, and pathlib.Path.glob().

We will also demonstrate how to filter files by type, list files recursively, and show you the best way to list all files in a directory. By the end of this article, you will have a solid understanding of the different methods available for listing files in a directory or folder in Python.

1. Quick Examples to List all Files of a Directory

Before diving into the details of each method, let's take a look at some quick examples. These Examples will provide you with a high-level overview of the different methods for listing files in a directory.


import os
import glob

# Using os.listdir()
dir_path = 'D:\Learning'
files = os.listdir(dir_path)

# Using os.scandir()
with os.scandir(dir_path) as entries:
    files = [entry.name for entry in entries if entry.is_file()]

# Using os.walk()
for root, dirs, files in os.walk(dir_path):
    for file in files:
        print(os.path.join(root, file))

# Using glob module
files = glob.glob(dir_path)

2. os.listdir() - List all Files in a Directory

The os.listdir() is a built-in Python function that returns a list of all the files and directories in a given directory. you may need to filter out directories if you only want to list files. You can use the os.path.isfile() function to check if each item in the list is returned by os.listdir() is a file or a directory.

In order to use this, you have to import the os module of Python.

See the following Example:


# Import os
import os

# List all files in a directory
path = 'D:\Learning'
files = []
for f in os.listdir(path):
    if os.path.isfile(os.path.join(path, f)):
        files.append(f)

print(*files, sep="\n")

The advantage of using os.listdir() is that it is a very lightweight function that is included in the Python standard library

You can also write


# Get only files in a directory
files = os.listdir(path)
files = [f for f in files if os.path.isfile(path+'/'+f)] 
print(*files, sep="\n")

3. os.scandir() Scan all Files in a Directory

The os.scandir() function also lists all file names in a directory and also various metadata about the file, such as its size, modification time, file type, and more. This is because the function returns an iterator of os.DirEntry objects.


# Use os.scandir() to list all files in the directory
with os.scandir(path) as entries:
    for entry in entries:
        # Print the name of the file
        print(entry.name)

You can also check more detail of the files. See the following Example.


# Use os.scandir() to list all files in the directory
with os.scandir(path) as entries:
    for entry in entries:
        # Print metadata about the file
        if entry.is_file():
            print(f'Size: {entry.stat().st_size} bytes')
            print(f'Modified: {entry.stat().st_mtime}')
            print(f'Created: {entry.stat().st_ctime}')
            print(f'File type: {entry.name.split(".")[-1]}')
            
        # Print metadata about the directory
        elif entry.is_dir():
            print(f'Created: {entry.stat().st_ctime}')
            print('---')

The path, I provided has only one file hence, it yields the following output.


Size: 656 bytes
Modified: 1676502870.324058
Created: 1676459004.3811111
File type: py
Size: 0 bytes
Modified: 1676502821.748931
Created: 1676502821.748931
File type: txt

4. os.walk() - Recursively List All Files

The os.walk() is a powerful method that allows you to recursively list all files and directories within a given directory and all its subdirectories. os.walk() provides a simple and efficient way to navigate and explore complex directory structures.


# It will list all the files inside 
# Directories and Subdirectories
for root, dirs, files in os.walk(path):
    for file in files:
        print(os.path.join(root, file))

The recursive nature of os.walk() can be a disadvantage in certain cases, such as when you only want to list files or directories at a specific level of the directory tree.

5. glob.glob() - Using Wildcard Patterns

The glob module provides a convenient way to search for files based on specific patterns in a directory. The glob module allows you to search for files based on a variety of patterns, including wildcard characters. This makes it easy to search for files with specific file extensions.


# List down All files
all_files= glob.glob('path')

for file in all_files:
    print(file)

5.1 Example: List only Text Files

In this example, glob.glob() is used to search for all .txt files in the specified directory. The * character is a wildcard that matches any string of characters.


# Print only text files 
txt_files = glob.glob('D:\Learning\*.txt')

for file in txt_files:
    print(file)

You can use this code to list any type of file in a directory. You can list all images by just replacing the .txt by .png or any other format.

5.2 Example: List all files Recursively

To list all files recursively in a directory using the glob module, we can use the ** pattern. The ** pattern matches any character, including the path separator (/), and can be used to recursively search through directories.


# Import modules
import glob
import os
files = glob.glob('D:\Learning\**', recursive=True)

for file in files:
    if not os.path.isdir(file):
        print(file)

6. pathlib.Path.glob() - Use the Pathlib Module

The pathlib module provides an object-oriented interface for working with the filesystem. It has a glob() method that can be used to list all files in a directory.

Advantage of using pathlib.Path.glob() is that it returns an iterator, which can be more memory-efficient than creating a list of all matching files.


# Import pathlib module
from pathlib import Path

# Get a path object for the directory to list
directory = Path('D:\Learning')

for file in directory.glob('*'):
    if file.is_file():
        print(file)

7. Best Way to List all Files in Directory

Depending on the specific requirements of your project, some methods may be better suited than others. Therefore, it's important to consider the specific needs of your project and choose the method that best suits them. Look at the following table for comparison.

| Method | Recursively lists all files | Filters out unwanted files | Metadata included | | --- | --- | --- | --- | | os.listdir() | No | No | No | | os.scandir() | Yes | Yes | Yes | | os.walk() | Yes | Yes | No | | glob.glob() | Yes | Yes | No | | pathlib.Path.glob() | Yes | Yes | Yes |

8. Summary and Conclusion

There are several methods to list all files in a directory in Python. Depending on the specific scenario and requirements, one method will be more suitable than the other. We hope that this article has helped you understand the differences between the methods and choose the best one for your needs. If you have any questions or comments, feel free to reach out.

Happy coding!!!