If you haven’t done so already, you may want to start by reading the Preface to the Computing Series: Software as a Force Multiplier, Sections 1-3.

“Everything” you need for ultra-fast desktop search

1. Everything(tm) is an ultra-fast desktop search utility that can scan through hundreds of thousands of files in milliseconds using a pre-built and real-time updated index.

“Everything” brings order to information growing at scale (documents, photographs, source code, spreadsheets, etc.), and tames the problem of proliferating folder trees.

We’ve all been in the scenario of searching through electronic documents for a document you know you prepared three, maybe four weeks prior… maybe it was longer… and now you can’t remember where you saved it… or in what format: was it a quickly written text file, a word document, a few paragraphs within One Note, on a desktop post-it note, or did you email yourself from your phone?… After trying different Windows searches in various recently used folders and looking through Word, Excel, and PDF files, and trying to remember possible filenames to search for, at some point you prepare mentally for the moment when you will give up the search and attempt to redo the missing work, salvaging as much of it as you can remember.

The general problem of wasted effort locating information we know we have, occurs more often than we’d like to admit. With “Everything“, it can be better.

Most Windows computer users appear to have accepted the apparent paradox that the fast search model that Google uses to index and organize the world’s information, is somehow not transferable to the personal desktop or laptop, despite the comparatively tiny amount of data on the local hard drive. But there is no paradox: niche tools exist for the specialist who knows this is a false contradiction. For ordinary users, the problem is decades of sub-standard Windows tools for file management and search from Microsoft’s operating systems. At the same time, the Windows user community, largely with limited technical knowledge, has not challenged Microsoft on the status quo and appears to have accepted the large penalties on average desktop search: 5 minutes or more to search for a file by attribute, maybe 20 minutes to an hour if searching through a large hard drive. And unfortunately for the end-user, the long search times don’t translate into higher success or satisfaction rates with desktop search.

“Everything” changes this. A 120MB hard drive with 500k files can be indexed in about 10 seconds, and a 1TB drive in less than a minute. Searching, thereafter, is a matter of seconds, no matter what the file type or where it is in the directory tree, as traversal is over a pre-built index. The key is how Everything maintains a real-time up-to-date catalog, which we will address below.

2. What is “Everything”?

Everything is software written by David Carpenter first released (v1.0.0) in 2008. In 2019, Everything received OSJoy’s Award for Best Desktop Search Engine for Windows 10. Whether a basic or advanced user, Everything offers enhanced capabilities for everyone. Ordinary users will appreciate the simple, spartan interface, built-in ability to preview contents of at least a dozen file types (text/src, html/mht, doc/docx, xls/xlsx, jpg/png/gif/bmp, mp4, and more). Power users will appreciate blazingly fast regular expression (regex) search, the ability rebuild indices to include more attributes (last modified date, creation date, contents, etc.), command line usage (using es.exe), and easy integration with the file managers they use most most (e.g. Total Commander through IPC (interprocess communication)).

3. How does Everything work?

When you first run “Everything” it will index all your files and folders (this should only happen once). It does this by reading the NTFS Master File Table (MFT) to build its initial database. This can take a few seconds for 100,000 files, and less than a minute for 1,000,000 files. Every run after this, “Everything” will update its database using the USN Change Journal, within a fraction of a second. The USN Change Journal is a log of file name changes maintained by the NTFS driver. More details are on the Everything FAQ page.

4. A bit of history

“Everything” originated in the Christmas week (Dec 22nd) of 2004, when David Carpenter started the project (build v0.1),see version history. First public release (v0.102) was three months later in March 2005. By February 2008, Everything was ready for official release (v1.0.0). It then had two further major releases in September 2008 (v1.1.4.301), and March 2009 (v1.2.1.371), after which it went into hibernation until August 2014 (v1.3.4.686). At this point it received upgrades to run on 64-bit machines, and a number of bug fixes and additional capabilities. After a second three year hibernation, another major upgrade was released in June 2017 to v.1.4.1.877, after which there have been five releases to v1.4.1.935 in February 2019.

5. Getting Started with Everything

1. Download from voidtools.  Select the portable 64-bit package.  While you’re at it, you should also download the command line utility es (for everything search) and the offline help file (everything.chm)
2. Unpack to c:\totalcmd\plugins\util_everything  (assuming you are using Total Commander and will want to integrate these two highly complementary tools)
3. Run Everything and configure from within:
   1. Press Alt+P to  enable preview pane
   2. Configure Everything through the menu:  Tools > Options…
   3. General > Check: (1) Run as Administrator (2) Everything Service
   4. UI > Check: (1) Run in background (2) Show tray icon (3) Single click tray
   5. View > Check: (1) Alternate row color (2) Show size in status bar
   6. Indices > (1) Database Location = same as unpack to c:\totalcmd\plugins\util_everything  (2) Compress database (5x space saving, and speed up on read/write), (3) Index recent changes, (4) Index file size (5) Index folder size (6) Fast size sort (7) uncheck everything else — unless you know you want it…

Alternatively: download everything.ini configuration file with all these settings already made (part of pre-configured Total Commander download, in c:\totalcmd\plugins\util_everything\

6. Integration with Total Commander (Power Users)

Total Commander (v9.0 or later) offers built-in integration with Everything for ultra-fast File Search (Alt+F7) using ed: and ev: search string prefixes, and Directory size calculation and ordering by size (SPACE for one directory, Alt+SHIFT+ENTER for all directories).

Getting the integration working is a bit tricky so I’ve pre-configured this in the Total Commander package available to download.

Setup using pre-configured Total Commander:

1. Download pre-configured TC with portable Everything pre-installed and pre-configured
2. Launch Everything before launching TC
3. Launch TC
4. Check Configuration > Options > Operations > is Everything box checked?  OK
5. Alt+F7 (Find Files): Everything is checked? 

Test to see that Everything is working:

1. Test: SPACE on a directory
2. Test: Go to c:\.  Ctrl+F6 (show details sorted by size).  ALT+SHIFT+ENTER – should almost instantaneously snap in all directory sizes (wait 2 seconds if first time usage)
3. Start a search for *.exe from c:\. Should take c.10 seconds and generate several thousands instances.

See Appendix B for behind-the-scenes look at how the hooks were placed in Total Commander to ensure seamless integration.

7. Final remarks

The fundamental technology for Everything is the Windows operating system file management mechanisms. It is surprising that Microsoft’s Windows Search is so considerably inferior, despite Everything being available (and open source) since 2008. Perhaps when users finally believe that information search and retrieval can be blazingly fast on a desktop, Microsoft will have a greater incentive to provide acceptable tools. In the meantime, those of you who have had enough can switch now, and experience the difference it makes when everything that’s on your personal hardware is, quite literally, at your fingertips in milliseconds!

---

Appendix A. Quickstart & References

1. Everything Official website
2. Quick start using Everything’s excellent online help
3. Using “Everything”, basic and advanced review, by Osjoy (Award for Best Desktop Search Engine for Windows 10)
4. How does it work? Everything’s comprehensive FAQ page
5. List of softwares that have integrated Everything.

---

Appendix B. Behind-the-Scenes: how to enable seamless integration in Total Commander

Total Commander (TC) communicates with Everything through an IPC (inter-process communication) channel.  This requires that both the Everything application (client) and the Everything service (background) are running before TC is run.  So we need TC to check whenever it starts up, whether these Everything components are already running, and if not, to launch them.

This can be done in three steps:

(new)Note to get rid of the User Access Control (UAC) warnings from Windows whenever Everything is started, you can use WinAero’s Shortcut Elevator. Once you have created the Elevated Shortcut (store it in the same folder as the un-elevated executable), then create a batch file (.bat) to point to the elevated link (one liner, the quotes must be present: “c:\totalcmd\plugins\util_everything\Everything(elevated).bat”), and then modify all Everything.exe path links below to point to the elevated batch file instead.
This is all done for you in the Total Commander Expansion pack.

(1) These are the wincmd.ini settings for Total Commander and Everything:

Everything=%COMMANDER_PATH%\plugins\util_everything\Everything.exe -first-instance (replace with .bat for elevation)
UseEverything=1
EverythingForSize=1


The option -first-instance prevents TC from attempting to launch new Everything instance, whenever a folder/file size is checked or when a file search is run.

(2) How to get TC to run this every time it starts up?

You’ll need to define a user command (em_startEverything) in a new INI file for TC called usercmd.ini

[em_startEverything]
button=%COMMANDER_PATH%\plugins\util_everything\Everything.exe (replace with .bat for elevation)
cmd=%COMMANDER_PATH%\plugins\util_everything\Everything.exe -first-instance (replace with .bat for elevation)
menu=Start Everything (ultra fast desktop search engine)


(3) Configure TC to run this command when the TC app launches.

Do this using the AutoRun plugin.  Follow the setup instructions there (using Color in TotCmd), and then add the following line into the AutoRun INI file:

CommandExec em_startEverything

That’s it! Now test as described above.

If all this seems too complicated, you can download pre-configured TC with Everything pre-installed and configured. Then follow the steps in the main body of the document.

---

Stay tuned for further articles in this series describing the software tools I use to speed things up and build working prototypes fast.

Articles in this series:

1. Efficient Computing: using Software as a “Force-Multiplier”
2. Total Commander (TC): a powerful two-panel file manager (“why explore when you can command?”)
3. Notepad++ (NPP): a programmer’s text editor.
4. Everything: an ultra-fast desktop search engine.

Richard Hamming (Applied Mathematician)

“Knowledge and productivity are like compound interest. The more you know, the more you learn; the more you learn, the more you can do; the more you can do, the more the opportunity — it is very much like compound interest.” – Richard Hamming, You and Your Research

---