Knocking up a simple and quick web server can be extremely useful without the need to install and configure a full fat web server. However, this isn't secure and will expose ALL files from the current directory executed in!
The official doc can be found here (v2) and here (v3).
NOTE: You will need to generate the certs for HTTPS version.
Just because! So sometimes you may need to patch a kernel source file, or a script, or maybe even your life who knows. But whatever the reason, creating a patch file and applying it are extremely easy.
Let's take the config from this post as an example.
Copy the file /etc/grub.d/10_linux to /etc/grub.d/10_linux_patched.
Open /etc/grub.d/10_linux_patched in your favourite editor and make the required changes (line 204), then save.
Original:
initrd=
for i in "initrd.img-${version}" "initrd-${version}.img" "initrd-${version}.gz" \
"initrd-${version}" "initramfs-${version}.img" \
"initrd.img-${alt_version}" "initrd-${alt_version}.img" \
"initrd-${alt_version}" "initramfs-${alt_version}.img" \
"initramfs-genkernel-${version}" \
"initramfs-genkernel-${alt_version}" \
"initramfs-genkernel-${GENKERNEL_ARCH}-${version}" \
"initramfs-genkernel-${GENKERNEL_ARCH}-${alt_version}"; do
if test -e "${dirname}/${i}" ; then
initrd="$i"
break
fi
done
Patched:
initrd=
for i in "initrd.img-${version}" "initrd-${version}.img" "initrd-${version}.gz" \
"initrd-${version}" "initramfs-${version}.img" \
"initrd.img-${alt_version}" "initrd-${alt_version}.img" \
"initrd-${alt_version}" "initramfs-${alt_version}.img" \
"initramfs-genkernel-${version}" \
"initramfs-genkernel-${alt_version}" \
"initramfs-genkernel-${GENKERNEL_ARCH}-${version}" \
"initramfs-genkernel-${GENKERNEL_ARCH}-${alt_version}"; do
if test -e "${dirname}/${i}" ; then
initrd="early_ucode.cpio ${rel_dirname}/$i"
break
else
initrd="early_ucode.cpio"
fi
done
If you just run a diff on the files, you see the changes.
The filenames that precede the '---' & '+++' are the files to be read '+++' and the file to be changed '---'. These 2 lines may also have a full or partial path to the files.
To apply the patch, just execute: patch < 10_linux.patch
If you now diff the 2 files, they should match ;)
So if not applying in the root of the files, we need to inform patch to omit the preceding segments.
example:
If the first 2 lines of the patch have a relative path.
Here we will get the firmware for the CPU loaded at boot. This helps the stability and efficiency of the processor.
Let's make sure we have the required options enabled in the kernel and rebuild if not.
General setup --->
[*] Initial RAM filesystem and RAM disk (initramfs/initrd) support
Processor type and features --->
<*> CPU microcode loading support
[*] Intel microcode loading support
If these were not enabled, enable them as above (not modules!) and recompile and install your kernel.
Next we'll install 2 packages required for this to work.
# emerge -av intel-microcode iucode_tool
These are the packages that would be merged, in order:
Calculating dependencies... done!
[ebuild R ] sys-firmware/intel-microcode-20161104::gentoo USE="split-ucode -initramfs -monolithic" 0 KiB
[ebuild R ] sys-apps/iucode_tool-2.1.1::gentoo 0 KiB
Mount your /boot filesystem and execute the following to generate the cpio archive.
You should see output like this with a different HEX value.
iucode_tool: system has processor(s) with signature 0x000206a7
iucode_tool: Writing selected microcodes to: /boot/early_ucode.cpio
Now we'll configure grub2 and install the new boot config.
Open the file /etc/grub.d/10_linux in your favourite editor and (at line ~204) make the following change.
From this:
initrd=for i in "initrd.img-${version}""initrd-${version}.img""initrd-${version}.gz"\"initrd-${version}""initramfs-${version}.img"\"initrd.img-${alt_version}""initrd-${alt_version}.img"\"initrd-${alt_version}""initramfs-${alt_version}.img"\"initramfs-genkernel-${version}"\"initramfs-genkernel-${alt_version}"\"initramfs-genkernel-${GENKERNEL_ARCH}-${version}"\"initramfs-genkernel-${GENKERNEL_ARCH}-${alt_version}";doiftest -e "${dirname}/${i}";theninitrd="$i"breakfidone
To:
initrd=
for i in "initrd.img-${version}" "initrd-${version}.img" "initrd-${version}.gz" \
"initrd-${version}" "initramfs-${version}.img" \
"initrd.img-${alt_version}" "initrd-${alt_version}.img" \
"initrd-${alt_version}" "initramfs-${alt_version}.img" \
"initramfs-genkernel-${version}" \
"initramfs-genkernel-${alt_version}" \
"initramfs-genkernel-${GENKERNEL_ARCH}-${version}" \
"initramfs-genkernel-${GENKERNEL_ARCH}-${alt_version}"; do
if test -e "${dirname}/${i}" ; then
initrd="early_ucode.cpio ${rel_dirname}/$i" break else initrd="early_ucode.cpio" fi done
Edit /etc/default/grub (at line ~59)
# Uncomment if you don't want GRUB to pass "root=UUID=xxx" parameter to kernel
GRUB_DISABLE_LINUX_UUID=true
And last off, install the new grub config:
grub-mkconfig -o /boot/grub/grub.cfg
And that is it!
If there is an update to sys-firmware/intel-microcode from this point that contains firmware for your CPU, just mount your boot FS and execute the iucode_tool command to generate a new cpio archive.
As always, first thing is first ;) Make sure you have the correct kernel options enabled.
File systems --->
[*] Dnotify support
[*] Network File Systems --->
<*> NFS client support
<*> NFS client support for NFS version 4
[*] NFS client support for NFSv4.1
<*> NFS server support
[*] NFS server support for NFS version 4
[*] NFSv4.1 server support for Parallel NFS (pNFS)
Depending on other options enabled in the kernel, version 3 may be auto selected and non-optional.
Once compiled and built. We can install the required apps. Make sure you enable to required USE flags.
emerge -av net-fs/nfs-utils
These are the packages that would be merged, in order:
Calculating dependencies... done!
[ebuild R ~] net-fs/nfs-utils-2.1.1::gentoo USE="libmount nfsidmap nfsv4 nfsv41 tcpd uuid -caps -ipv6 -kerberos -nfsdcld (-selinux)"
SERVER:
Now we'll create the directory structure. The first line is the virtual root followed by each filesystem.
Systemd although very good, has caused problems for logging (for me). Systemd does not play nice with logging applications such as Splunk. The reason for this is that systemd takes over syslog and stores all log data in its journald system which uses tmpfs (RAM) until flushed to disk in its proprietary format. This is to make log data more secure.
Here is how I got around that so that I could analyse my logs. It's pretty straight forward and isn't too involved.
Configure journald.conf
I have set the following options in my config and everything else is commented out.
The journald config above will now send everything to syslog which by default will store in /var/log/messages. To split out specific logs, you'll need to tell syslog what to do with them. Below is a basic config to split a few logs that I'm interested in. Some apps may use their own logger, so be aware of this.
# cat /etc/syslog-ng/syslog-ng.conf
@version: 3.7
# $Id$
#
# Syslog-ng default configuration file for Gentoo Linux
# https://bugs.gentoo.org/show_bug.cgi?id=426814
@include "scl.conf"
options {
threaded(yes);
chain_hostnames(no);
# The default action of syslog-ng is to log a STATS line
# to the file every 10 minutes. That's pretty ugly after a while.
# Change it to every 12 hours so you get a nice daily update of
# how many messages syslog-ng missed (0).
stats_freq(43200);
# The default action of syslog-ng is to log a MARK line
# to the file every 20 minutes. That's seems high for most
# people so turn it down to once an hour. Set it to zero
# if you don't want the functionality at all.
mark_freq(3600);
};
source src {
system();
internal();
};
destination messages { file("/var/log/messages"); };
# By default messages are logged to tty12...
destination console_all { file("/dev/tty12"); };
# ...if you intend to use /dev/console for programs like xconsole
# you can comment out the destination line above that references /dev/tty12
# and uncomment the line below.
#destination console_all { file("/dev/console"); };
# iptables log
destination firewall { file("/var/log/firewall.log"); };
filter f_firewall { program("iptables") or match("Dropped" value(MESSAGE)); };
log { source(src); filter(f_firewall); destination(firewall); flags(final); };
# ssh log
destination sshd { file("/var/log/sshd.log"); };
filter f_sshd { program("^sshd$"); };
log { source(src); filter(f_sshd); destination(sshd); flags(final); };
# named log
destination named { file("/var/log/named.log" owner(named) group(named) perm(0600) dir_perm(0700)); };
filter f_named { program("^named$"); };
log { source(src); filter(f_named); destination(named); flags(final); };
# dhcp log
destination dhcpd { file("/var/log/dhcpd.log"); };
filter f_dhcpd { program("^dhcpd$"); };
log { source(src); filter(f_dhcpd); destination(dhcpd); flags(final); };
# spamd log
destination spamd { file("/var/log/spamd.log"); };
filter f_spamd { program("^spamassassin$") or program("^/usr/sbin/spamd$"); };
log { source(src); filter(f_spamd); destination(spamd); flags(final); };
# ALWAYS AT THE END
log { source(src); destination(messages); };
log { source(src); destination(console_all); };
First thing is to ensure the correct options are set in the kernel.
IP sets support the following type of sets:
bitmap:ip
The bitmap:ip set type uses a memory range, where each bit represents one IP address and can store up to 65535 (B-class network) entries. You can store same size network addresses in this kind of sets as well and an IP address will be in the set if the network address it belongs to can be found in the set.
bitmap:ip,mac
The bitmap:ip,mac set type uses a memory range, where each 8 bytes represents one IP and a MAC addresses. A bitmap:ip,mac set type can store up to 65535 (B-class network) IP addresses with MAC.
bitmap:port
The bitmap:port set type uses a memory range, where each bit represents one TCP/UDP port. A bitmap:port type of set can store up to 65535 ports.
hash:ip
The hash:ip set type uses a hash to store IP addresses where clashing is resolved by storing the clashing elements in an array and, as a last resort, by dynamically growing the hash. Same size network addresses can be stored in an hash:ip type of set as well.
hash:net
The hash:net set type also uses a hash to store CIDR netblocks, which may be of different sizes. The same techique is used to avoid clashes as at the hash:ip set type.
hash:ip,port
The hash:ip,port is similar to hash:ip but you can store IP address and protocol-port pairs in it. TCP, SCTP, UDP, UDPLITE, ICMP and ICMPv6 are supported with port numbers/ICMP(v6) types and other protocol numbers without port information.
hash:ip,port,ip
You can store IP address, port number, and IP address triples in an hash:ip,port,ip type of set.
hash:ip,port,net
You can store IP address, port number and network address triples in this kind of set.
hash:net,port
The set type supports to store network address and port number pairs.
hash:net,iface
In this kind of set one can store network address and interface name pairs.
list:set
In a list:set kind of set you can store other sets; it is like an ordered union of different sets.
-*- Networking support --->
Networking options --->
[*] Network packet filtering framework (Netfilter) --->
<*> IP set support --->
(256) Maximum number of IP sets
<*> bitmap:ip set support
<*> bitmap:ip,mac set support
<*> bitmap:port set support
<*> hash:ip set support
<*> hash:ip,mark set support
<*> hash:ip,port set support
<*> hash:ip,port,ip set support
<*> hash:ip,port,net set support
<*> hash:ip,mac set support
<*> hash:mac set support
<*> hash:net,port,net set support
<*> hash:net set support
<*> hash:net,net set support
<*> hash:net,port set support
<*> hash:net,iface set support
<*> list:set set support
If you're having to now add this, you'll obviously need to reboot once you have built and installed the updated kernel.
Now let's install ipset.
# emerge -av ipset
These are the packages that would be merged, in order:
Calculating dependencies... done!
[ebuild R ] net-firewall/ipset-6.29::gentoo USE="-modules"
Ipset has the modules flag set by default. With the above kernel config, the build will fail as I haven't used modules DUH! So just disable the flag. If you've build the kernel with modules instead, just leave as is.
So on to the fun stuff. Ipset has a pretty good help section and the manpage is detailed :)
I use systemd, so there is no service to restart for iptables or ipset.
Obviously you'll have iptables configured and running :) By default I block everything and only have specific ports open to the world. But that doesn't mean we don't need ipsets. I'm going to use ipset to block spiders (crawlers) that I don't want or that bypass my robots.txt. I could limit the blocking to specific ports eg 80 & 443, but I'm just going to block everything. Just because I can ;) I've also set a timeout (optional) so the IP's can be rotated and the sets don't grow too big.
So let's create an ipset. I only want to block CIDR, so I don't need to add port, mac etc etc.
ipset create netSpiders hash:net timeout 86400
You can add IPs in here also, but I like to keep things clean so I also have:
ipset create ipSpiders hash:ip timeout 86400
NOTE: If you haven't rebooted into the ipset enabled kernel, you'll get the error:
ipset v6.29: Kernel error received: set type not supported
You can list the set and any rules within by executing:
# ipset list
Name: spiders
Type: hash:net
Revision: 6
Header: family inet hashsize 1024 maxelem 65536 timeout 86400
Size in memory: 368
References: 0
Number of entries: 0
Members:
As you can see, we have nothing defined yet. So let's add some spiders. Unfortunately it's per CIDR.
su - apache -c "php /path/to/nextcloud/occ twofactorauth:disable username"
This will disable 2FA regardless of it it is enabled in the GUI. You will need to enable 2FA by running the command again, but with enable instead.
Delete the undeleteables.
I had a situation where a user had ~30Gb of data in their nextcloud trash which they could not delete. There were also files/directories in their active files that they also could not delete. No errors in the logs, just unable to delete :( The work around is to manually delete them from the server and then run the following command.
su - apache -c "php /path/to/nextcloud/occ files:scan <user>"
This removes invalid file references from the database.
Here I will show how to configure VSFTPD for basic authentication so that we have a base working daemon. Then we will build on that by implementing SSL and then virtual users.
Obviously first thing is first ;) If you haven't already, install vsftpd.
The above will tell vsftpd to not allow any anonymous connections eg mandatory login. What and where to log. What IP and port to listen on. To lock users into their home directory (defined in /etc/passwd).
The bottom 3 lines userlist_* we define so we don't grant all local users ftp access. If you answer YES to userlist_deny, the user list will deny any users listed in the file and allow everything else.
$ ftp cdstealer.com
Connected to cdstealer.com (<IP>).
220 (vsFTPd 3.0.2)
Name (cdstealer.com:user1):
530 Please login with USER and PASS.
SSL not available
331 Please specify the password.
Password:
230 Login successful.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp>
SSL
So let's get some encryption so we aren't transmitting plain text credentials.
Generate a self signed cert.
# openssl req -x509 -nodes -days 3650 -newkey rsa:4096 -keyout /etc/ssl/apache2/vsftp.pem -out /etc/ssl/apache2/vsftp.pem
Generating a 4096 bit RSA private key
..........................................................................................++
....++
writing new private key to '/etc/ssl/apache2/vsftp.pem'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:GB
State or Province Name (full name) [Some-State]:Somewhere nice
Locality Name (eg, city) []:Leeds
Organization Name (eg, company) [Internet Widgits Pty Ltd]:cdstealer.com
Organizational Unit Name (eg, section) []:
Common Name (e.g. server FQDN or YOUR name) []:cdstealer.com
Email Address []:
The openssl command I've used, generates a 4096bit encrypted cert (this is good) that is valid for 10 years. :)
Execute the following to remove unwanted access to the cert.
chmod 600 /etc/ssl/apache2/vsftp.pem
Add this section to your /etc/vsftpd/vsftpd.conf file.
Let's test and make sure everything is still working.
$ ftp cdstealer.com
Connected to cdstealer.com (<IP>).
220 (vsFTPd 3.0.2)
Name (cdstealer.com:user1):
234 Proceed with negotiation.
[SSL Cipher AES128-SHA]
331 Please specify the password.
Password:
230 Login successful.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp>
GREAT! So now we have a secure FTP service running.
In the interest of security, I'm not comfortable having system user accounts, even though we have secured things with only specific users able to ftp. I believe that standard ftp authentication does not support encryption for system users, but does for virtual users?
This is a little more involved than having standard system users :(
So, add the following to your /etc/vsftpd/vsftpd.conf file.
Here we define that virtual users get the same permissions as the ftp system user, are unable to browse outside their directory, use a PAM database for credentials and define custom settings per user.
NOTE: You may notice that the file extension is missing from the path in /etc/pam.d/vsftpd. This is intentional as PAM automatically adds the .db suffix.
You will need to add the users from the database to the access list file.
/etc/vsftpd/vsftpd.user_list
For your convenience, I've written a user management script, here :)
Define custom settings.
Create the directory which will store the configs. We defined this earlier as user_config_dir=/etc/vsftpd/virtualUsers.
mkdir /etc/vsftpd/virtualUsers
Create a user config.
vi /etc/vsftpd/virtualUsers/user1
write_enable=NO
local_root=/FTP/user1
chroot_local_user=YES
dirlist_enable=YES
download_enable=YES
Here we set the users CHROOT directory, deny write permissions and allow downloading. Options here (not all) override specific options defined in the main config.
If you get the following when logging in or listing a directory, it maybe due to the user directory not existing or not having permission.
ssl_getc: SSL_read failed -1 = 0
421 Service not available, remote server has closed connection
Login failed.
No control connection for command: Success
The following error may occur on ftp clients with vsftpd 3.0.x:
500 OOPS: priv_sock_get_cmd
This is caused by seccomp filter sanboxing, which is enabled by default on amd64. To workaround this issue, disable seccomp filter sanboxing:
Add the following line to /etc/vsftpd/vsftpd.conf.
When prompted, enter the necessary information for creating a CSR by using the conventions shown in the following table.
Note: The following characters cannot be used in the Organization Name or theOrganizational Unit: < > ~ ! @ # $ % ^ * / \ ( ) ?.,&
DN field
Explanation
Example
Common Name
The fully qualified domain name for your web server. This must be an exact match.
If you intend to secure the URL https://www.yourdomain.com, then your CSR’s common name must be www.yourdomain.com. If you plan to get a wildcard certificate, make sure to prefix your domain name with an asterisk, for example: *.domain.com.
Organization Name
The exact legal name of your organization. Do not abbreviate your organization name.
domain.com
Organizational Unit
Section of the organization.
IT
City or Locality
The city where your organization is legally located.
Wellesley Hills
State or Province
The state or province where your organization is legally located. Do not use an abbreviation.
Massachusetts
Country
The two-letter ISO abbreviation for your country.
US
Warning: Leave the challenge password blank (press Enter).
