For businesses interested in deploying and running diskover, please visit https://diskoverdata.com to learn more.

Join the conversation, get support on diskover Slack.

Create an account to download and run diskover community edition.

file system crawler

File System Crawler

diskover crawls your local, NFS/SMB, or cloud storage servers and scrapes file/directory meta data into Elasticsearch.

Learn More

analyze metadata

Visualize Your Data

Identify old and unused files and give better insights into data change, duplicate files and wasted disk space.

Learn More

open-source

Open Source

Built using Python + PHP + HTML5 + Javascript + D3.js. diskover runs on Linux, macOS, and Windows.

Learn More

News/ Updates

diskover v2 will be released soon (Q1 2021), please sign up and register at https://diskoverdata.com/diskover/ for updates and join diskover Slack. v1 is end of life and is no longer supported.

This is the first tool I've found that can index 7m files/2m directories in under 20 min

-- linuxserver.io community member

diskover is an open source file system crawler and data management and visualization software that uses Elasticsearch to index and manage data across heterogeneous storage systems. Using diskover, you are able to more effectively search and organize files and system administrators are able to manage storage infrastructure, efficiently provision storage, monitor and report on storage use, and effectively make decisions about new infrastructure purchases.

As the amount of file data generated by businesses continues to expand, the stress on expensive storage infrastructure, users and system administrators, and IT budgets continues to grow.

Using diskover, users can identify old and unused files and give better insights into data change, file duplication and wasted space. diskover crawls local file-systems, NFS/SMB and cloud storage, etc.

Screenshots (v1)

See data change on your file system and identify hot spots
diskover-web hotdirs
Visualize your file system using one of the many analytics
diskover-web filetree
Tag files and directories using default and custom tags, export file lists
diskover-web tagging
Use the built-in rest-api to assist with tagged data cleanup/moving
diskover-web tags
Use pre-made or create custom smart searches
diskover-web tagging
Find duplicate files taking up disk space
diskover-web dupes
Quickly search all your storage servers
diskover-web file search
Use Elasticsearch query syntax to find files and directories
diskover-web file search es query
Run in VMware Run in Amazon AWS Run in Docker
diskover vmware diskover aws diskover docker
diskover can be set up in VMware or on bare-metal, modular design allows you to run crawlers on bare-metal, ES and diskover-web in VMware or any way you like. diskover works on AWS using EC2 and Elasticsearch instances. Crawlers can run locally and push file system meta data into your AWS ES cluster. Run diskover and diskover-web containers anywhere. Docker install instructions can be found on diskover github.

diskover worker bots crawling file system (gource videos)