Choosing the Right File Structures

Last updated on 2026-07-31 | Edit this page

Overview

Questions

  • Why does folder organisation matter?
  • What makes a good folder structure?
  • How can I identify problems in an existing file structure?
  • How can I move, rename, and delete folders using a file browser?
  • How can I organise a project so that it is easier to understand and maintain?

Objectives

By the end of this episode, learners will be able to:

  • Navigate the file browser using a mouse.
  • Identify issues with folder names in a poorly organised project.
  • Rename folders using a graphical file browser.
  • Identify issues with folder organisation.
  • Move folders using a graphical file browser.
  • Delete unnecessary folders using a graphical file browser.
  • Design a folder structure that supports collaboration and reproducibility.

File Structures

In the previous episode we located the dataset that was handed over by our departed collaborator. Unfortunately, finding the data is only the first challenge.

The project structure itself is difficult to understand - the files include

legacy_dataset/
│
├── miscellaneous/
│   ├── cat_pic.jpg
│   ├── important_note.txt
│   └── Untitled.csv
│
├── New Folder
│   ├── big_file.txt
│   ├── Image One.png
│   ├── image_1.png
│   ├── image_10.png
│   ├── image_2.png
│   ├── image_8.png
│   ├── image_80.png
│   ├── image_81.png
│   ├── image_9.png
│   ├── image_93.png
│   ├── IMG_000.png
│   ├── IMG_001.png
│   ├── IMG_002.png
│   ├── ...
│   ├── IMG_018.png
│   ├── IMG_019.png
│   ├── June second image.png
│   ├── test.png
│   ├── Untitled 1.csv
│   └── New Folder (2)/
│       └── backup_copy.txt
│
├── penguins
│   ├── palmer_penguins.csv
│   ├── penguin958.jpg
│   └── penguin958.png
│
├── project_data_2025/
│   ├── data_final_v3_LAST_ONE.xlsx
│   ├── analysis_12-05-24.txt
│   ├── data_final_v4_revised_jerrys_comments.xlsx
│   ├── data_final_v2.xlsx
│   ├── script.py
│   ├── Alice_Jones_passport.pdf
│   ├── john_smith_passport.pdf
│   ├── script.py
│   └── analysis_12-05-24.txt
│   └── .processed/
│        └── REAL_ACTUAL_DATA_DO_NOT_DELETE.csv
│
├── processing_data
│   ├── 04-11-2025_penguin_data.csv
│   ├── 10-Nov-26_penguin_withtheirnames_and_weatherreport.txt
│   └── ...
│
└── RAW_DATA_!!!
    ├── Data!@#$.csv
    ├── 03_04_26_results.csv
    ├── raw_images_TEMP/
    └── ...

and more!

Imagine joining this project six months from now.

Could you quickly answer:

  • Where is the raw data?
  • Which files should be analysed?
  • Which files are temporary?
  • Which files should be preserved?
  • Which files can be deleted?

For most people, the answer is “probably not”. Even if details are stored in lab notes, these can easily be misplaced.

Before we worry about individual files, we need to make sure the overall project structure makes sense.

Why Folder Structures Matter

Most research projects start small with only a few files:

project/
├── data.csv
├── notes.docx
└── report.docx

At this stage organisation may seem unnecessary. However, projects rarely stay this simple. More data is added or revised, code is written, plots are generated, and papers/reports/theses are written… and revised.

Callout

As projects grow we often accumulate:

  • Raw data
  • Processed data
  • Analysis outputs
  • Figures
  • Documentation
  • Draft manuscripts
  • Collaborator comments
  • Software scripts
  • Temporary files
  • Meeting minutes
  • Downloaded software
  • Software or equipment manuals
  • Various notes
  • Talks and presentations

A project that starts with three files may eventually contain hundreds or thousands.

Without a clear structure, researchers often experience:

  • Time wasted searching for files
  • Duplicate work
  • Confusion over what files contain
  • Difficulty onboarding collaborators
  • Difficulty reproducing previous work

Good folder structures make projects easier to:

  • Navigate
  • Understand
  • Collaborate on
  • Maintain over time

A useful rule of thumb is

A collaborator should be able to understand where files belong without asking you.

Callout

The best practice is to plan for expansion rather than reorganising files later, which can lead to incorrect file paths and references in files such as software scripts or lab notes.

Establish a possible file structure early in the project, while being flexible if new folders or files need to be added or moved.

Some of this organisation depends on personal preference. For example,

  • Do you want to store a seminar or conference presentation with the project, or with other talks elsewhere?
  • Should a paper, along with generated figures, be kept together with the rest of the project?

If these files are stored elsewhere, can you to find the origin of figures and other results when you write a paper in six months time? If you move a folder, will your file references still work?

Challenge

What’s Wrong With This Structure?

Working in groups of 2–3:

Spend 5 minutes exploring the inherited dataset.

Identify as many problems as you can.

Consider:

  • Folder names
  • Folder organisation
  • Folder depth
  • Duplication
  • Clarity
  • Consistency

Write down your observations.

After 5 minutes, bring the class back together and discuss.

Common observations often include:

  • Default names such as New Folder
  • Unclear names such as miscellaneous
  • Use of special characters (RAW_DATA_!!!)
  • Hidden data locations
  • Mixed file types within the same folder
  • Folders whose purpose is unclear
  • Temporary files mixed with project files

Principles of Good Folder Structures

There is no single correct folder structure because different projects have different requirements.

However, most successful structures share common characteristics.

  • Meaningful
  • Consistent
  • Shallow
  • Categorical
  • Able to archive old files
Discussion

Common Characteristics

After looking at the Poor examples in each subsection below, try and come up with some good examples before clicking the Good tab.

Meaningful

Folder names should communicate purpose. For example,

Callout

If somebody joined the project tomorrow, would they understand what belongs in this folder?

Consistent

Consistency is more important than perfection. Choose a particular style and use it throughout the project. This approach makes the structure easier to understand because all folders follow the same convention. For example,

Shallow

Deeply nested structures can become difficult to navigate.

Avoid situations like:

Project/
└── Data/
    └── NewData/
        └── New Folder/
            └── Updated/
                └── Updated Again/
                    └── Results/

Having to click through many layers increases the chance of losing track of files.

As a general guideline:

  • 3–4 levels deep is often sufficient
  • Create more folders horizontally before creating more levels vertically

Avoid using the folder hierarchy to encode too much information. For example

Project/
└── Data/
    └── Raw/
        └── 2025/
            └── 05/
                └── 06/
                    └── 01/
                        └── Newcastle/
                            └── A_roads/
                                └── UserID12345/
                                └── UserID53546/
                            └── Minor_roads/
                                └── UserID35790/
                                └── UserID53546/
                        └── Gateshead/
                    └── 02/
                        └── Sunderland/
                    └── 03/
                        └── Results/

At best, this approach results in a lot of additional clicks in a file explorer or typing in a terminal/scripts. It can also make analyses unwieldy if you need to collect data from many different subfolders (e.g., if you want to analyse all of Newcastle’s data from the example above).

At worst, file metadata, such as city and datetime in this example, can be irrevocably lost if files are moved from their original location. As such, metadata should ideally be stored in the files themselves or in a reference table with the filenames, rather than in the folder structure.

Categorical

A common source of confusion is mixing different types of data and files together so that plots, data, reports, analyses, and scripts are all stored in the same folder. Instead, group related materials in the same folder.

This structure makes it easier to know where new files belong and find existing files, even if you’re new to the project.

Caution

Know where any private/secure data is to minimise the chance of GDPR violations.

Archive Rather Than Hoard

Many researchers hesitate to delete anything. As a result, they accumulate folders full of obsolete material. Instead, consider creating an archive folder:

archive/

This approach allows old material to be retained without cluttering the active project structure.

We will discuss data retention in a later episode.

Moving Files and Folders

Now that we know what good structures look like, we need the practical skills to improve existing structures.

The first operation is moving content.

How to Move a Folder

Renaming Folders

Renaming is one of the simplest improvements we can make to the file organisation. However, be aware that renaming can have unintended consequences if existing files depend on those names; for example, you may need to update folder references in the project’s documentation or code.

Discussion

Challenge

Choose a better name for the folder in the Poor example, then see one potential better name in the Good tab.

How to Rename a Folder

Caution

Be careful when renaming folders that are used by software or analysis scripts. Changing a folder name may break links elsewhere in a project.

Additionally, project documentation may become out-of-date and confusing if it references the original folder structure.

Deleting Unnecessary Folders

Sometimes the simplest improvement is removing unused material.

Consider:

New Folder (2)

If this folder is empty or no longer required, it may be reasonable to remove it.

How to Delete a Folder

Data deleted using the above methods are not fully removed from the system; they are first moved to the recycle bin. You can permanently delete the file by emptying the bin. Alternatively, files in the recycle bin can be restored and “put back” to their original location. On some systems, the bin permanently deletes files after they have been in the bin a certain amount of time (e.g., 30 days).

Caution

Check your data policies on data deletion, particularly if handling private data and your files are not encrypted. We will discuss data security in a later lesson.

Deleting a file on your computer will not always remove the file from automated backups (e.g., files stored in Newcastle University’s Microsoft OneDrive); this behaviour will depend on your syncing settings. Confirm that these settings support the backup behaviour you need for your project.

Designing a Better Structure

At this stage we know:

  • What good folder names look like
  • How to move folders
  • How to rename folders
  • How to remove unnecessary folders

The next step is deciding what folder structure you need for your project.

Challenge

Choosing a Folder Structure

For one of your projects, consider possible useful folders and how you would arrange them. What folder structures would work well? How would you divide up files?

Try drawing out three possible folder hierarchies for your project and discuss the pros/cons of the different approaches with your neighbours.

There is no one correct answer. It will vary depending on your project or personal preferences. One possible structure might be:

project/
├── data/
│   ├── raw/
│   └── processed/
│
├── analysis/
│
├── documentation/
│
├── outputs/
│
└── archive/

Other possible folders would be:

  • code
  • figures folder inside the analysis folder or outputs folder
  • temp for temporary files you know you do not wish to keep etc.

The most important part of the design is that each folder has a clear purpose.

Discussion

What Else Would You Change?

Looking at the inherited dataset:

  • Which folders would you rename?
  • Which folders would you move?
  • Which folders would you remove?
  • Are there any new folders you would create?

Compare ideas with a neighbour.

Challenge

Repair the Project Structure

Working individually or in pairs:

  1. Rename unclear folders.
  2. Move folders into more logical locations.
  3. Remove any unnecessary empty folders.
  4. Create a structure that clearly separates:
    • Data
    • Documentation
    • Analysis
    • Outputs

You do not need to create a perfect structure.

Focus on making the project easier to understand.

There is no single correct solution.

A reasonable result might:

  • Rename RAW_DATA_!!! to raw_data
  • Replace miscellaneous with more specific folders
  • Remove unnecessary empty folders
  • Create dedicated locations for documentation and analysis
  • Reduce unnecessary nesting

Looking Ahead

We have improved the folder structure, but many files still have confusing names.

In the next episode we will focus on file naming conventions and explore how good file names make datasets easier to understand and process.

Key Points
  • Folder structures should help people find and understand data.
  • Use meaningful, consistent folder names.
  • Separate different types of project content into dedicated folders.
  • Avoid excessive nesting.
  • File Explorer allows folders to be moved, renamed, and deleted without using the command line.
  • Consistency is usually more important than any specific organisational scheme.
  • A good folder structure should be understandable by collaborators and by your future self.