Jtdylktuy

I've been renting a VPS for half a year now (educational purposes) and I've been trying to learn as much as possible about keeping it secure.

Recently, it was compromised and I suspect that it has been used as someone's proxy for ~a week before I realized. I had logs from users 'anonymous' and 'nobody' logging in and out via SSH, and the CPU usage was off the chart - literally.

In any case, I reinstalled it and I re-applied everything I knew to test if it'll happen again, and within the short 24 hours after the reinstall, I think it did.

Here are the respective logs from /var/log/auth.log that make me feel paranoid:

Oct 31 06:30:21 vultr su[24157]: Successful su for nobody by root

Oct 31 06:30:21 vultr su[24157]: + ??? root:nobody

Oct 31 06:30:21 vultr su[24157]: pam_unix(su:session): session opened for user nobody by (uid=0)

Oct 31 06:30:21 vultr systemd: pam_unix(systemd-user:session): session opened for user nobody by (uid=0)

Oct 31 06:30:21 vultr systemd-logind[503]: New session 40 of user nobody.

Oct 31 06:30:24 vultr su[24157]: pam_unix(su:session): session closed for user nobody

Oct 31 06:30:24 vultr systemd-logind[503]: Removed session 40.

I wasn't the one to authenticate at 6:30 in the morning, so naturally I'm worried that I messed up again somewhere ...

(note: .bash_history of root shows nothing suspicious, and as far as I know user nobody doesn't have a .bash_history - correct me if I'm wrong)

Password authentication is disabled for SSH, only SSH key authentication is possible, which is why I'm really confused about what to try next since someone has still managed to get access (I think).

I've read this article about a phpMyAdmin exploit where the attacker acquired access to the 'nobody' user. However I don't think it applies to my case, because according to my Apache logs there haven't been any attempts to access the phpMyAdmin page, not to mention the article is dated 2010 and my phpMyAdmin page is not even accessible at the moment.

The type of requests, however, that Apache receives are worrying me a bit, here's an example (from /var/log/apache2/access.log):

192.99.144.140 - - [31/Oct/2015:03:43:48 +0000] "PROPFIND /webdav/ HTTP/1.1" 405 569 "-" "WEBDAV Client"

185.25.151.159 - - [31/Oct/2015:03:59:35 +0000] "GET http://testp2.czar.bielawa.pl/testproxy.php HTTP/1.1" 404 460 "-" "Mozilla/5.0 (Windows NT 5.1; rv:32.0) Gecko/20100101 Firefox/31.0"

61.228.95.69 - - [31/Oct/2015:09:07:39 +0000] "CONNECT 126mx00.mxmail.netease.com:25 HTTP/1.0" 405 536 "-" "-"

185.25.151.159 - - [31/Oct/2015:09:15:13 +0000] "GET http://testp4.pospr.waw.pl/testproxy.php HTTP/1.1" 404 457 "-" "Mozilla/5.0 (Windows NT 5.1; rv:32.0) Gecko/20100101 Firefox/31.0"

whereas I'd be expecting something more like this:

my ip - - [31/Oct/2015:14:47:58 +0000] "GET / HTTP/1.1" 200 589 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.130 Safari/537.36"

I don't understand what is accomplished by attempting to

"GET http://testp4.pospr.waw.pl/testproxy.php"

from my server. There is no such file or directory in /var/www/html

I'll list all the things I did to try and stay secure after the reinstall. Please point out if you think I've done something incorrectly or if I've not done something at all.

What I did:

1. Fresh install of Ubuntu 15.04


2. Generate public and private key pair


3. Add my public key to my authorized_keys file on my server


4. 
   

   Change these settings in /etc/ssh/sshd_config, thus disabling password authentication and only allowing SSH key authentication only

   
   
   

   PermitRootLogin without-password
   
   RSAAuthentication yes
   
   PubkeyAuthentication yes
   
   PasswordAuthentication no
   
   

   
   



5. Reboot



6. 
   

   Install the things I need, in this order:

   
   
   

   zip 
   
   unzip 
   
   apache2
   
   mysql-server
   
   php5 libapache2-mod-php5
   
   openjdk-7-jdk
   
   gcc
   
   g++
   
   screen
   
   vsftpd
   
   auditd
   
   

   
   



7. Remove the default /var/www/html/index.html



8. 
   

   Configure vsftpd to work (I have a strong password for FtpUser). I followed this tutorial from DigitalOcean

   
   
   

   mkdir /home/proj
   
   groupadd ftp-users
   
   chown root:ftp-users /home/proj
   
   chown root:ftp-users /var/www
   
   useradd -g ftp-users -d /home/proj FtpUser
   
   chown FtpUser /home/proj
   
   passwd FtpUser    (add strong password)
   
   

   
   
   

   I changed these settings in /etc/vsftpd.conf

   
   
   

   anonymous_enable=NO
   
   local_enable=YES
   
   write_enable=YES
   
   chroot_local_user=NO
   
   pam_service_name=ftp
   
   

   
   



9. Reboot

Everything else that I didn't mention is at it's default, all software installed is up to date.

Please let me know if the logs I showed earlier are something to worry about.
I'd also be very grateful if you could tell me whether my configuration is incorrect and what I can do to improve my security further. Additionally, if you know any good articles on the matter that would be very helpful in the long run.

Run hiawatha webserver as a reverse proxy in front of your webserver. It will block exploits such as this (they will be blocked as "garbage") in the logs:

91.196.50.33|Sat 19 Mar 2016 21:12:15 +0000|GET http://testp3.pospr.waw.pl/testproxy.php HTTP/1.1

Host: testp3.pospr.waw.pl

Proxy-Connection: Keep-Alive

User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:32.0) Gecko/20100101 Firefox/31.0

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

Accept-Language: pl,en-US;q=0.7,en;q=0.3

Accept-Encoding: gzip, deflate

Run the webserver & proxy inside separate lxc containers to further isolate the processes.

Use the chroot feature built into php-fpm.

Do NOT make a shell available inside the chroot

Stealth your ssh port.

Mount your /var/www/public_html as noexec nosuid nodev.