How to use a shell command to only show the first column and last column in a text file?
I need some help to figure out how to use the sed command to only show the first column and last column in a text file. Here is what I have so far for column 1:
My feeble attempt at getting the last column to show as well was:
However this takes the first column and last column and merges them together in one list. Is there a way to print the first column and last columns clearly with sed and awk commands?
6 Answers 6
Almost there. Just put both column references next to each other.
Also note that you don’t need cat here.
Also note you can tell awk that the column separators is | , instead of blanks, so you don’t need sed either.
As per suggestions by Caleb, if you want a solution that still outputs the last field, even if there are not exactly eight, you can use $NF .
Also, if you want the output to retain the | separators, instead of using a space, you can specify the output field separators. Unfortunately, it’s a bit more clumsy than just using the -F flag, but here are three approaches.
You can assign the input and output field separators in awk itself, in the BEGIN block.
You can assign these variables when calling awk from the command line, via the -v flag.
You are using awk anyway:
Just replace from the first to last | with a | (or space if you prefer):
Note that though there’s no sed implementation where | is special (as long as extended regular expressions are not enabled via -E or -r in some implementations), \| itself is special in some like GNU sed . So you should not escape | if you intend it to match the | character.
If replacing with space and if the input may already contain lines with only one | , then, you’ll have to treat that specially as |.*| won’t match on those. That could be:
(that is make the .*| part optional) Or:
If you want the first and eighth fields regardless of the number of fields in the input, then it’s just:
(all those would work with any POSIX compliant utility assuming the input forms valid text (in particular, the sed ones will generally not work if the input has bytes or sequences of bytes that don’t form valid characters in the current locale like for instance printf ‘unix|St\351phane|Chazelas\n’ | sed ‘s/|.*|/|/’ in a UTF-8 locale)).
How to get the first column of comm output?
So I’m trying to get the first column of comm output using awk . I read that Tab was used as a separator for comm so I did:
With comm-result.txt containing the output of:
But this doesn’t seem to work.
This commend takes also the space character as a separator and I get weird results when my files contains multiple spaces.
How can i only get the first column from comm ?
3 Answers 3
«So I’m trying to get the first column of comm output»
The first column of the » comm file1 file2 » output contains lines unique to the file1 . You can skip the post-processing by simply calling comm with -2 (suppress lines unique to file2 ) and -3 (suppress lines that appear in both files).
However, if you have no choice but to process a pre-run output of comm then as Carl mentioned, cut would be an option:
However, this result in empty lines for cases where column 1 is empty. To deal with this, perhaps awk may be more suitable:
grep to search data in first column
I have a text file with two columns.
I want to search for product starts from «Abc» and ends with «def» then for those entries I want to add Cost. I have used :
but it is not working
/^Abc.*def/
3 Answers 3
Use awk. cat myfile | awk ‘
Thanks edited. Do you want the command like this?
If you can use awk , try this:
- awk reads each line and matches the first column with a regular expression (regex)
- The first column has to start with Abc, followed by anything (zero or more times), and ends with def
- If such match is found, add 2nd column to SUM variable
- After reading all lines print the variable
Grep extracts each line that starts with Abc, followed by anything, followed by def, followed by anything, followed by a number (zero or more times) to end. Those lines are fed/piped to awk. Awk just increments SUM for each line it receives. After reading all lines received, it prints the SUM variable.
Not the answer you’re looking for? Browse other questions tagged linux ubuntu grep or ask your own question.
Related
Hot Network Questions
Subscribe to RSS
To subscribe to this RSS feed, copy and paste this URL into your RSS reader.
site design / logo © 2020 Stack Exchange Inc; user contributions licensed under cc by-sa. rev 2020.9.18.37632
Substring only the first column in awk
I want to take first 500 characters of the first column and leave the rest of the columns as it is, using awk.
This is what I am trying to do:
But I get syntax error:
2 Answers 2
You’re trying to pass (invalid) bash subscript syntax to awk, I’m not sure how you expect that to work.
1 is just a way of returning true so that awk prints the line, substr() is the actual call that does the substring. From the documentation:
substr(string, start, length)
This returns a length-character-long substring of string, starting at character number start. The first character of a string is character number one.
For example, substr(«washington», 5, 3) returns «ing». If length is not present, this function returns the whole suffix of string that begins at character number start. For example, substr(«washington», 5) returns «ington». This is also the case if length is greater than the number of characters remaining in the string, counting from character number start.
If you need to keep the field separator, set OFS appropriately.
How to get the first column of every line from a CSV file?
How do get the first column of every line in an input CSV file and output to a new file? I am thinking using awk but not sure how.
6 Answers 6
It will split each input line in the file data.txt into different fields based on , character (as specified with the -F ) and print the first field (column) to stdout.
Input
Code
Format
This can be achieved using grep :
perl -F, -lane ‘print $F[0]’ data.txt > data2.txt
These command-line options are used:
- -n loop around every line of the input file
- -l removes newlines before processing, and adds them back in afterwards
- -a autosplit mode – split input lines into the @F array. Defaults to splitting on whitespace.
- -e execute the perl code
- -F autosplit modifier, in this case splits on ,
If you want to modify your original file in-place, use the -i option:
perl -i -lane ‘print $F[0]’ data.txt
If you want to modify your original file in-place and make a backup copy:
perl -i.bak -lane ‘print $F[0]’ data.txt
If your data is whitespace separated rather than comma-separated:
how to print first n column and a column which matches the pattern?
I have a tab delimited file with n rows and m columns, I want to print the first three column and search for a pattern and print that column if present. I tried to search and print in sed but unable to do it to print first 3 column and then search for the pattern.
example I have file like
output I want is (if pattern I search is ‘col6’ for example)
3 Answers 3
You can iterate through the fields when the awk is reading the first line and determine which field col6 is present,
What it does?
NR==1 If the current number of records(lines) read is 1, then iterate through NF number of fields
- if ($i == «col6») if the current column is equal to the string we search, we save it in variable column .
print $1, $2, $3, column ? $column : «» Prints the first three fields. The column field is printed only if it is set, if not prints empty «».
Print the first column
I want to print column 1 of this file. I used this command: awk ‘
DATA
Output:
Desired Output:
4 Answers 4
What I can see is that your columns are delimited by two space.
Since this seems to be a fixed-width column, you can just cut the corresponding characters. The widest column Alanine, aspartate and glutamate metabolism seems to be 44 characters wide, so:
As the second column obviously repeats the beginning of the first column, I take this as criterion for the cut with sed , thus it does not depend on the column width:
First pattern is the repeated part, backreferenced as \1 at the end of the line. You could add ;s/ *$// to remove the trailing spaces if they bother you.
Building upon muru’s answer that the column is specified with fixed width, using egrep command with option -o will allow you to print just the matched (non-empty) parts of a matching line specified by the search pattern. By default, however, entire line will be printed.