Jtdylktuy

Yesterday evening I lost all of my project, or to be more exact, the directory tree named f/ that was in my home directory.

I restored from a backup.

A few minutes ago it disappeared again (fortunately for me, I started backing up with every rebuild since then and I rebuild often so no losses this time). I'm not aware of any executable I could have run or written myself that could've done this.

Since the issue seems to be isolated to ~/f/, I think it's some buggy script that wanted to do rm -rf and instead did rm -r f.

How can I find the culprit?

Assuming rm is indeed involved and it's /bin/rm:

1. sudo ln /bin/rm /bin/rm-real


2. 
   

   Create a log directory writable to anyone:

   
   
   

   mkdir /tmp/rm-logdir
   
   chmod a+w /tmp/rm-logdir
   
   

   
   


3. 
   

   Write a wrapper script (named script) like this:

   
   
   

   #!/bin/sh
   
   pstree -spa "$$" >> "/tmp/rm-logdir/$$"
   
   exec /bin/rm-real "$@"
   
   

   
   



4. Adjust ownership and permissions of the script so it can mimic rm (non-POSIX fast way:chmod --reference=/bin/rm script; sudo chown --reference=/bin/rm script).



5. 
   

   Replace /bin/rm with the script:

   
   
   

   sudo mv script /bin/rm
   
   

   
   
   

   From now on anything that calls /bin/rm will be logged to a file in /tmp/rm-logdir.

   
   



6. Wait for the problem to occur again.



7. 
   

   Examine files in /tmp/rm-logdir. One of them will hopefully contain something like this:

   
   
   

   systemd,1
   
     `-tmux: server,2652
   
         `-bash,8605
   
             `-rm,15100 /bin/rm -r f
   
                 `-pstree,15101 -spa 15100
   
   

   
   
   

   with command line arguments and PIDs; note the last but one line. The above is in fact an example from my tests in Kubuntu, where I run rm -r f in interactive bash. Another example (not triggered directly by me) was:

   
   
   

   systemd,1
   
     `-kdeinit5,1679
   
         `-ksmserver,1700
   
             `-rm,16400 /bin/rm /home/kamil/.config/session/kate_10d8d5de64000154299531000000017000020_1542995310_707537
   
                 `-pstree,16401 -spa 16400
   
   

   
   
   

   It proves other tools (ksmserver in this case) use the wrapper script.

   
   

Notes:

• 
  pstree is not required by POSIX, it may be unavailable. You can use ps, you can read information directly from /proc.


• In multi-user environment the solution is not safe. Anyone can write to the log directory. This is deliberately set to capture anyone's rm invocations in a lazy, quick and dirty way. But then anyone can spoof the results.

To revert:

sudo mv /bin/rm-real /bin/rm

(note this will overwrite the script, so if there's no other copy of it, the script itself will be lost).