When developing server-side software it’s important that you use logs. Just as importantly though, you need to know how to “rotate” your logs, or prevent them from overtaking your server space.
Since we use Ubuntu 13.10 currently we use the logrotate feature to help set up our log files and automatically manage log rotations so that our logs won’t overtake the server. It’s fairly simple, but not everyone knows how to do this. Logrotate should be installed on your server, if it’s not, first install it.
sudo apt-get update
sudo apt-get install logrotate
To verify it’s now installed run the following
Now cd into the logrotate.d folder to see the logrotate scripts
This folder contains your logrotate scripts. Since we use Node.JS and Forever to keep our servers running we set up this as our configuration file:
This creates your file, and then copy paste the following into your file, where the first line is the location of your forever log file
Run this to verify your new logrotate script is running
This will tell you what logs are being rotated, and your logs should show up there now!
Here’s a breakdown of what the logrotate commands mean:
Old versions of log files are compressed with gzip by default. See also nocompress.
Specifies which command to use to compress log files. The default is gzip. See alsocompress.
Specifies which command to use to uncompress log files. The default is gunzip.
Specifies which extension to use on compressed logfiles, if compression is enabled. The default follows that of the configured compression command.
Command line options may be passed to the compression program, if one is in use. The default, for gzip, is “-9” (maximum compression).
Make a copy of the log file, but don’t change the original at all. This option can be used, for instance, to make a snapshot of the current log file, or when some other utility needs to truncate or pare the file. When this option is used, the create option will have no effect, as the old log file stays in place.
Truncate the original log file in place after creating a copy, instead of moving the old log file and optionally creating a new one, It can be used when some program can not be told to close its logfile and thus might continue writing (appending) to the previous log file forever. Note that there is a very small time slice between copying the file and truncating it, so some logging data might be lost. When this option is used, the create option will have no effect, as the old log file stays in place.
create mode owner group
Immediately after rotation (before the postrotate script is run) the log file is created (with the same name as the log file just rotated). mode specifies the mode for the log file in octal (the same as chmod(2)), owner specifies the user name who will own the log file, and groupspecifies the group the log file will belong to. Any of the log file attributes may be omitted, in which case those attributes for the new file will use the same values as the original log file for the omitted attributes. This option can be disabled using the nocreate option.
Log files are rotated every day.
Postpone compression of the previous log file to the next rotation cycle. This has only effect when used in combination with compress. It can be used when some program can not be told to close its logfile and thus might continue writing to the previous log file for some time.
Log files are given the final extension ext after rotation. If compression is used, the compression extension (normally .gz) appears after ext.
Rotate the log file even if it is empty, overiding the notifempty option (ifempty is the default).
Reads the file given as an argument as if it was included inline where the include directive appears. If a directory is given, most of the files in that directory are read in alphabetic order before processing of the including file continues. The only files which are ignored are files which are not regular files (such as directories and named pipes) and files whose names end with one of the taboo extensions, as specified by the tabooext directive. The includedirective may not appear inside of a log file definition.
When a log is rotated out-of-existence, it is mailed to address. If no mail should be generated by a particular log, the nomail directive may be used.
When using the mail command, mail the just-rotated file, instead of the about-to-expire file.
When using the mail command, mail the about-to-expire file, instead of the just-rotated file (this is the default).
If the log file is missing, go on to the next one without issuing an error message. See alsonomissingok.
Log files are rotated the first time logrotate is run in a month (this is normally on the first day of the month).
Old versions of log files are not compressed with gzip. See also compress.
Do not copy the original log file and leave it in place. (this overrides the copy option).
Do not truncate the original log file in place after creating a copy (this overrides thecopytruncate option).
New log files are not created (this overrides the create option).
Do not postpone compression of the previous log file to the next rotation cycle (this overrides the delaycompress option).
Don’t mail old log files to any address.
If a log file does not exist, issue an error. This is the default.
Logs are rotated in the same directory the log normally resides in (this overrides the olddiroption).
Run prerotate and postrotate scripts for every script which is rotated (this is the default, and overrides the sharedscripts option).
Do not rotate the log if it is empty (this overrides the ifempty option).
Logs are moved into directory for rotation. The directory must be on the same physical device as the log file being rotated. When this option is used all old versions of the log end up indirectory. This option may be overriden by the noolddir option.
The lines between postrotate and endscript (both of which must appear on lines by themselves) are executed after the log file is rotated. These directives may only appear inside of a log file definition. See prerotate as well.
The lines between prerotate and endscript (both of which must appear on lines by themselves) are executed before the log file is rotated and only if the log will actually be rotated. These directives may only appear inside of a log file definition. See postrotate as well.
The lines between firstaction and endscript (both of which must appear on lines by themselves) are executed once before all log files that match the wildcarded pattern are rotated, before prerotate script is run and only if at least one log will actually be rotated. These directives may only appear inside of a log file definition. See lastaction as well.
The lines between lastaction and endscript (both of which must appear on lines by themselves) are executed once after all log files that match the wildcarded pattern are rotated, after postrotate script is run and only if at least one log is rotated. These directives may only appear inside of a log file definition. See lastaction as well.
Log files are rotated times before being removed or mailed to the address specified in a maildirective. If count is 0, old versions are removed rather then rotated.
Log files are rotated when they grow bigger then size bytes. If size is followed by M, the size if assumed to be in megabytes. If the k is used, the size is in kilobytes. So size 100, size 100k, and size 100M are all valid.
Normally, prescript and postscript scripts are run for each log which is rotated, meaning that a single script may be run multiple times for log file entries which match multiple files (such as the /var/log/news/* example). If sharedscript is specified, the scripts are only run once, no matter how many logs match the wildcarded pattern. However, if none of the logs in the pattern require rotating, the scripts will not be run at all. This option overrides the nosharedscripts option.
This is the number to use as the base for rotation. For example, if you specify 0, the logs will be created with a .0 extension as they are rotated from the original log files. If you specify 9, log files will be created with a .9, skipping 0-8. Files will still be rotated the number of times specified with the count directive.
tabooext [+] list
The current taboo extension list is changed (see the include directive for information on the taboo extensions). If a + precedes the list of extensions, the current taboo extension list is augmented, otherwise it is replaced. At startup, the taboo extension list contains .rpmorig, .rpmsave, ,v, .swp, .rpmnew, and ~.
Log files are rotated if the current weekday is less then the weekday of the last rotation or if more then a week has passed since the last rotation. This is normally the same as rotating logs on the first day of the week, but it works better if logrotate is not run every night.