Portable way to get file size (in bytes) in shell?
On Linux, I use stat —format=»%s» FILE , but Solaris I have access to doesn’t have stat command. What should I use then?
I’m writing Bash scripts, and can’t really install any new software on the system.
I’ve considered already using:
But neither of these looks sensible — running Perl just to get file size? Or running 2 commands to do the same?
14 Answers 14
wc -c (short for word count, -c prints the byte count) is a portable, POSIX solution. Only the output format might not be uniform across platforms as some spaces may be prepended (which is the case for Solaris).
Do not omit the input redirection. When the file is passed as an argument, the file name is printed after the byte count.
I was worried it wouldn’t work for binary files, but it works OK on both Linux and Solaris. You can try it with wc -c . Moreover, POSIX utilities are guaranteed to handle binary files, unless specified otherwise explicitly.
I ended up writing my own program (really small) to display just the size. More information here: http://fwhacking.blogspot.com/2011/03/bfsize-print-file-size-in-bytes-and.html
The two most clean ways in my opinion with common Linux tools are:
But I just don’t want to be typing parameters or pipe the output just to get a file size, so I’m using my own bfsize.
Even though du usually prints disk usage and not actual data size, GNU coreutils du can print file’s «apparent size» in bytes:
But it won’t work under BSD, Solaris, macOS, .
Finally I decided to use ls, and bash array expansion:
it’s not really nice, but at least it does only 1 fork+execve, and it doesn’t rely on secondary programming language (perl/ruby/python/whatever)
Cross platform fastest solution (only uses single fork() for ls, doesn’t attempt to count actual characters, doesn’t spawn unneeded awk, perl, etc).
Tested on MacOS, Linux — may require minor modification for Solaris:
If required, simplify ls arguments, and adjust offset in $<__ln[3]>.
Note: will follow symlinks.
BSDs have stat with different options from the GNU coreutils one, but similar capabilities.
This works on macOS (tested on 10.12), FreeBSD, NetBSD and OpenBSD.
When processing ls -n output, as an alternative to ill-portable shell arrays, you can use the positional arguments, which form the only array and are the only local variables in standard shell. Wrap the overwrite of positional arguments in a function to preserve the original arguments to your script or function.
This splits the output of ln -dn according to current IFS environment variable settings, assigns it to positional arguments and echoes the fifth one. The -d ensures directories are handled properly and the -n assures that user and group names do not need to be resolved, unlike with -l . Also, user and group names containing whitespace could theoretically break the expected line structure; they are usually disallowed, but this possibility still makes the programmer stop and think.
If you use find from GNU fileutils:
Unfortunately, other implementations of find usually don’t support -maxdepth , nor -printf . This is the case for e.g. Solaris and macOS find .
You can use find command to get some set of files (here temp files are extracted). Then you can use du command to get the file size of each file in human readable form using -h switch.
You first Perl example doesn’t look unreasonable to me.
It’s for reasons like this that I migrated from writing shell scripts (in bash/sh etc.) to writing all but the most trivial scripts in Perl. I found that I was having to launch Perl for particular requirements, and as I did that more and more, I realised that writing the scripts in Perl was probably a more powerful (in terms of the language and the wide array of libraries available via CPAN) and more efficient way to achieve what I wanted.
Note that other shell-scripting languages (e.g. python/ruby) will no doubt have similar facilities, and you may want to evaluate these for your purposes. I only discuss Perl since that’s the language I use and am familiar with.
if you have Perl on your Solaris, then use it. Otherwise, ls with awk is your next best bet, since you don’t have stat or your find is not GNU find.
There is a trick in Solaris I have used, if you ask for the size of more than one file it returns just the total size with no names — so include an empty file like /dev/null as the second file:
eg command fileyouwant /dev/null
I can’t rememebr which size command this works for ls/wc/etc — unfortunately I don’t have a solaris box to test it.
How can I see the size of files and directories in linux? [closed]
Want to improve this question? Update the question so it’s on-topic for Stack Overflow.
Closed 21 days ago .
How can I see the size of files and directories in Linux? If use df -m , then it shows the size of all the directory at the top level, but, for the directories and files inside the directory, how do I check the size?
22 Answers 22
Use ls command for files and du command for directories.
Checking File Sizes
ls command will not list the actual size of directories(why?). Therefore, we use du for this purpose.
Checking Directory sizes
Including -h option in any of the above commands (for Ex: ls -lh * or du -sh ) will give you size in human readable format ( kb , mb , gb , . )
For more information see man ls and man du
There is du command.
Size of a directory and/or file:
—apparent-size command line switch makes it measure apparent sizes (what ls shows) rather than actual disk usage.
Use ls -s to list file size, or if you prefer ls -sh for human readable sizes.
For directories use du , and again, du -h for human readable sizes.
Using this command you’ll see the apparent space of the directory and true space of the files and in details the names of the files displayed, besides the size and creation date of each.
There is also a great ncdu utility — it can show directory size with detailed info about subfolders and files.
Installation
Usage
Just type ncdu [path] in the command line. After a few seconds for analyzing the path, you will see something like this:
Delete the currently highlighted element with d , exit with CTRL + c
Go to the chosen directory and execute:
You’ll see like that:
ls -l —block-size=M will give you a long format listing (needed to actually see the file size) and round file sizes up to the nearest MiB.
If you want MB (10^6 bytes) rather than MiB (2^20 bytes) units, use —block-size=MB instead.
If you don’t want the M suffix attached to the file size, you can use something like —block-size=1M. Thanks Stéphane Chazelas for suggesting this.
This is described in the man page for ls; man ls and search for SIZE. It allows for units other than MB/MiB as well, and from the looks of it (I didn’t try that) arbitrary block sizes as well (so you could see the file size as number of 412-byte blocks, if you want to).
Note that the —block-size parameter is a GNU extension on top of the Open Group’s ls, so this may not work if you don’t have a GNU userland (which most Linux installations do). The ls from GNU coreutils 8.5 does support —block-size as described above.
HowTo: Linux / Unix See File Size Command
a] ls command – list directory contents.
Tutorial details | |
---|---|
Difficulty | Easy (rss) |
Root privileges | No |
Requirements | ls/du/stat |
Time | Less than a one minute |
b] du command – estimate file space usage.
c] stat command – display file or file system status.
Examples
To determine the size of a file called /bin/grep, enter:
In the above output example, the 175488 is the size of the file. For a more user friendly output, pass the -h option to the ls command:
In the above output example, the 172K is the size of the file. The du command provides the same output in a more user friendly way and it hides all other details too:
du -h /bin/grep
Sample outputs:
Finally, stat command also provide file size:
stat /bin/grep
Sample outputs:
The following commands are executed on Apple OS X Unix operating systems to file out the file size of the /usr/bin/vim binary file:
$ ls -l /usr/bin/vim
$ ls -lh /usr/bin/vim
$ stat -x /usr/bin/vim
Sample outputs:
Fig.01: Finding out file size using various command line options on OS X Unix OS
Get total size of a list of files in UNIX
I want to run a find command that will find a certain list of files and then iterate through that list of files to run some operations. I also want to find the total size of all the files in that list.
I’d like to make the list of files FIRST, then do the other operations. Is there an easy way I can report just the total size of all the files in the list?
In essence I am trying to find a one-liner for the ‘total_size’ variable in the code snippet below:
6 Answers 6
You should simply be able to pass $file_list to du :
- -c display a total
- -h human readable (i.e. 17M)
du will print an entry for each file, followed by the total (with -c ), so we use tail -1 to trim to only the last line and cut -f 1 to trim that line to only the first column.
Methods explained here have hidden bug. When file list is long, then it exceeds limit of shell comand size. Better use this one using du:
find produces null ended file list, du takes it from stdin and counts. this is independent of shell command size limit. Of course, you can add to du some switches to get logical file size, because by default du told you how physical much space files will take.
But I think it is not question for programmers, but for unix admins 🙂 then for stackoverflow this is out of topic.
This code adds up all the bytes from the trusty ls for all files (it excludes all directories. apparently they’re 8kb per folder/directory)
Note: Execute as root. Result in megabytes.
ls -l | tr -s ‘ ‘ | cut -d ‘ ‘ -f is something I use a lot.
The 5th field is the size. Put that command in a for loop and add the size to an accumulator and you’ll get the total size of all the files in a directory. Easier than learning AWK. Plus in the command substitution part, you can grep to limit what you’re looking for (^- for files, and so on).
The method provided by @Znik helps with the bug encountered when the file list is too long.
However, on Solaris (which is a Unix), du does not have the -c or —total option, so it seems there is a need for a counter to accumulate file sizes.
In addition, if your file names contain special characters, this will not go too well through the pipe (Properly escaping output from pipe in xargs ).
Based on the initial question, the following works on Solaris (with a small amendment to the way the variable is created):
The output is in KiB.
The problem with du is that it adds up the size of the directory nodes as well. It is an issue when you want to sum up only the file sizes. (Btw., I feel strange that du has no option for ignoring the directories.)
In order to add the size of files under the current directory (recursively), I use the following command:
How it works: it lists all the files recursively ( «R» ), including the hidden files ( «a» ) showing their file size ( «l» ) and without ordering them ( «U» ). (This can be a thing when you have many files in the directories.) Then, we keep only the lines that start with «-» (these are the regular files, so we ignore directories and other stuffs). Then we merge the subsequent spaces into one so that the lines of the tabular aligned output of ls becomes a single-space-separated list of fields in each line. Then we cut the 5th field of each line, which stores the file size. The awk script sums these values up into the sum variable and prints the results.