File System Crawler
diskover crawls your local, NFS/SMB, or cloud storage servers and scrapes file/directory meta data into Elasticsearch.
Visualize Your Data
Identify old and unused files and give better insights into data change, duplicate files and wasted disk space.
diskover v2 will be released soon (Q1 2021), please sign up and register at https://diskoverdata.com/diskover/ for updates and join diskover Slack. v1 is end of life and is no longer supported.
-- linuxserver.io community memberThis is the first tool I've found that can index 7m files/2m directories in under 20 min
diskover is an open source file system crawler and data management and visualization software that uses Elasticsearch to index and manage data across heterogeneous storage systems. Using diskover, you are able to more effectively search and organize files and system administrators are able to manage storage infrastructure, efficiently provision storage, monitor and report on storage use, and effectively make decisions about new infrastructure purchases.
As the amount of file data generated by businesses continues to expand, the stress on expensive storage infrastructure, users and system administrators, and IT budgets continues to grow.
Using diskover, users can identify old and unused files and give better insights into data change, file duplication and wasted space. diskover crawls local file-systems, NFS/SMB and cloud storage, etc.
|Run in VMware||Run in Amazon AWS||Run in Docker|
|diskover can be set up in VMware or on bare-metal, modular design allows you to run crawlers on bare-metal, ES and diskover-web in VMware or any way you like.||diskover works on AWS using EC2 and Elasticsearch instances. Crawlers can run locally and push file system meta data into your AWS ES cluster.||Run diskover and diskover-web containers anywhere. Docker install instructions can be found on diskover github.|
diskover worker bots crawling file system (gource videos)