Awk Or Sed Help

Hi all ,
I have a requirement where I have a file. Contents of the file are :
#comments
VAR="abg"
RES=123
#comments
IC6790ABG="https://www.abc.com"
IC5678-vg="https://www.bhy.com"
IC-gy_567:78="https://www.gyt.com"
#comments
The variable names can not have characters like - , : so
in this file I have to find words starting with IC and replace characters like - ,:
I want to change only the variable name , not the whole line.
I have used SED command

sed -i '/^IC/s/[^0-9 a-z A-Z _]*//g' file

when I am using this command , it is replacing the whole line
output becomes :

#comments
VAR="abg"
RES=123
#comments
IC6790ABGhttpswwwabccom
IC5678vghttpswwwbhycom
ICgy56778httpswwwgytcom
#comments


But I want the output like this :

#comments
VAR="abg"
RES=123
#comments
IC6790ABG="https://www.abc.com"
IC5678vg="https://www.bhy.com"
ICgy_56778="https://www.gyt.com"
#comments

How can I get the desired output , thanks for your help in advance .

hi guys

i am trying to find the "size" of a "block" of data in LARGE data files, the example below test_data.txt is very simplified. by "size" i mean the difference in line numbers of a block, and the "size" will be constant throughout the file so

1234 6.600000 4321
1234 8.500000 4321
1234 1.800000 4321
1234 2.300000 4321
1234 8.500000 4321
1234 2.800000 4321

if i define a block as whenever i find 8.500000 in the second column, then in the example the the block size would be 3 becasue 8.500000 occurs on the 5th line and on the 2nd. right now i am using

Code:

 grep -n "8.500000" test_data.txt | cut -f1 -d:

and/or

Code:

 awk '/8.500000/ {print FNR}' test_data.txt

obviously i don't remeber how to tag text as code?

btw, the grep command is much much faster

both of these commands give an entire list (long list of number for files greater than a gig) of line numbers which i then have to subtract one from another to come up with 3 in the example. not that i'm opposed to doing math, but i would think awk or grep should be able to do this for me

ideas?

tabby

Code:

PREFIX=192.168.1
OCTET=1
while [ $OCTET -lt "255" ]; do
    echo -en "Pinging ${PREFIX}.${OCTET}..."
    ping -c1 -w1 ${PREFIX}.${OCTET} > /dev/null 2>&1
    if [ "$?" -eq "0" ]; then
       echo "OK"
    else
       echo "Failed"
    fi
    let OCTET=$OCTET+1
done

My question is, in the penultimate line, why is it OCTET, and not $OCTET, with a dollar before it, meaning, the variable?

What if it were written:
Code:

let $OCTET=$OCTET+1

Like in C++ (I think) or Python or whatever?

Hi

I am using this command to get some info about bots/spiders from my Centos server access.log file:

Code:

grep 'spider\|bot' access.log | sort -u -f >> bots.txt

Result is like this (i know pingdom is not bad):

Code:

141.101.105.102 - - [28/Mar/2015:01:59:56 +0200] "GET / HTTP/1.1" 200 24194 "-" "Pingdom.com_bot_version_1.4_(http://www.pingdom.com/)"
141.101.105.158 - - [28/Mar/2015:02:09:56 +0200] "GET / HTTP/1.1" 200 24260 "-" "Pingdom.com_bot_version_1.4_(http://www.pingdom.com/)"
141.101.105.102 - - [28/Mar/2015:02:19:56 +0200] "GET / HTTP/1.1" 200 24277 "-" "Pingdom.com_bot_version_1.4_(http://www.pingdom.com/)"
108.162.215.53 - - [27/Mar/2015:23:13:21 +0200] "GET /user/74595-tery1/?tab=idm HTTP/1.1" 200 3905 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
108.162.215.53 - - [27/Mar/2015:23:11:59 +0200] "GET /user/275904-ktlk21/ HTTP/1.1" 200 3805 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
108.162.215.75 - - [27/Mar/2015:23:21:31 +0200] "GET /user/74595-tery1/?tab=topics HTTP/1.1" 200 13588 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"

Is there any command that can remove duplicate lines if the ip and the user-agent is the same on each line?

To get something like:

Code:

141.101.105.102 - - [28/Mar/2015:01:59:56 +0200] "GET / HTTP/1.1" 200 24194 "-" "Pingdom.com_bot_version_1.4_(http://www.pingdom.com/)"
141.101.105.158 - - [28/Mar/2015:02:09:56 +0200] "GET / HTTP/1.1" 200 24260 "-" "Pingdom.com_bot_version_1.4_(http://www.pingdom.com/)"
108.162.215.53 - - [27/Mar/2015:23:11:59 +0200] "GET /user/275904-ktlk21/ HTTP/1.1" 200 3805 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
108.162.215.75 - - [27/Mar/2015:23:21:31 +0200] "GET /user/74595-tery1/?tab=topics HTTP/1.1" 200 13588 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"

Or if there is no way for this then to get only one line (even if different ip's exist for each user agent) like:

Code:

141.101.105.102 - - [28/Mar/2015:01:59:56 +0200] "GET / HTTP/1.1" 200 24194 "-" "Pingdom.com_bot_version_1.4_(http://www.pingdom.com/)"
108.162.215.53 - - [27/Mar/2015:23:11:59 +0200] "GET /user/275904-ktlk21/ HTTP/1.1" 200 3805 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"

Hello,

They gave me this:
Code:

#!/bin/bash

# Write error message on stderr and die
function die() {
  echo "$@" >&2
  exit 1
}

# Load var from properties files
function load() {
  for propertie in "$@" ; do
    [[ -f "${propertie}" ]] || die "load() : \"${propertie}\" don't exist !"
    while read ; do
echo "${REPLY%=*}"
echo "${REPLY#*=}"
      [[ -n "${REPLY%%=*}" && -z "${REPLY#*=}" ]] || die "load() : Variable \"${REPLY%%=*}\" is empty!"
      eval "${REPLY%%=*}"=\'"${REPLY#*=}"\'
    done < "${propertie}"
  done
}

load "manage_srcds.conf"
echo "SRCDS_TICKRATE=$SRCDS_TICKRATE"
echo "SRCDS_SCREEN=$SRCDS_SCREEN"
echo "SRCDS_SRCDS_GAME_NAME=$SRCDS_GAME_NAME"
echo "SRCDS_MAXPLAYERS=$SRCDS_MAXPLAYERS"

but, return:
Quote:
###################################################
###################################################
load() : Variable "###################################################" is empty!

cat manage_srcds.conf

I just wish that there was no empty variable excluding those who do not have integer.

Best regards,

I am not sure how to ask this, sorry.

if I had a code like this

Code:

# grep -a ": " md5list.txt | cut -f2,3 -d

How can I run the command basename for each line of the output?

basename {(grep -a ": " md5list.txt | cut -f2,3 -d )}


EDIT: A little more clarity on what im doing:
I didn't realize that 'md5sums' was a link to a nice formatted page. So I copied all packages here and put them into a text file. I decided to write a script that put all of these in that format.

So basically, even though I have already ran the md5sum -c 'md5sum-list' I still want to finish this small project because I am learning a ton.

https://code.google.com/p/wavelet1d/...ar.gz&can=2&q=

The above link is where I downloaded *.cpp(s) and the libraries.

If you "untar" the package, in "examples" directory, there are some demonstration files. What I wanted to do was to make an executable file out of "wavedemo1.cpp".

I modified the code in "wavedemo1.cpp"; Code:

#include "wavelet2d.h"

to Code:

#include "wavelet2s.h"

, then placed the header file "wavelet2s.h" (from /wavelib/src/linuxstatic) and the modified "wavedemo1.cpp" into my working directory.

Inside the working directory, I ran the following command

Code:

gcc -L/home/mario/wavelet/wavelib/linuxstatic -lwavelet2s wavedemo1.cpp -o wavedemo1

Then I get the following error messages.

/usr/bin/ld: skipping incompatible /home/mario/wavelet/wavelib/linuxstatic/libwavelet2s.a when searching for -lwavelet2s
/usr/bin/ld: cannot find -lwavelet2s
collect2: error: ld returned 1 exit status

I'm not sure what went wrong with my approach. This is my first time trying to use libraries on linux and it's giving me really tough times.

I would appreciate your help, please let me know if you need more details to explain things.

Thanks.

Hello everyone,

I recently started using "function" and "case" in linux but I came across a slight problem

My code:

Code:

function Return
{
echo $1
}

case "$1" in 
 ""       ) echo "give parameters please";;
 [0-9]*   ) echo "No numbers please";;
 *        ) Return;;

So whenever the first parameter is a simple word I would like it returned, but it seems like there is no returned string (empty space)

Also I know I can just write "echo $1" instead of Return, but I'd like to use a function for this.

Can anyone tell me why the function doesn't return $1? Thanks!

Hi Guys,
i have this Code:

uid=0(root) gid=0(root)

i would like to have the word "root" only from the first field, and later from the second field

i use the command
Code:

[root@v1-6 ~]# cat test-uid |cut -d"(" -f1
brms2
uid=0

[root@v1-6 ~]# cat test-uid |cut -d")" -f1
brms2
uid=0(root

it cut the word after the -d, how can i make it to cut the word BEFORE the -d

expected should be:
cut -d"(" -f1
root)

then i can proceed to eliminate the ) later on.
hope this is not confusing you guys

Thanks,

Hi all,

Been reading on stdin, stdout and stderr and encounter 2 questions, hope gurus here can advise.

0 = stdin
1 = stdout
2 = stderr

Code:

Sun Dec 21 03:53:42 SGT 2014 > cat test5.sh
#!/bin/bash

echo "Please enter value for name :"
read name
echo "Your name is $name."

echo "Next echo will be a syntax error"
ehco

Code:

Sun Dec 21 03:53:46 SGT 2014 > test5.sh 1> output.txt 2> error.txt
Noob

Sun Dec 21 03:54:56 SGT 2014 > cat output.txt
Please enter value for name :
Your name is Noob.
Next echo will be a syntax error

Sun Dec 21 03:55:23 SGT 2014 > cat error.txt
/home/alan/scripts/test5.sh: line 8: ehco: command not found
Sun Dec 21 03:55:26 SGT 2014 >

Which so far all is good and the correct way to output everything including error to a single file is

Code:

Sun Dec 21 03:59:14 SGT 2014 > test5.sh > output.txt 2>&1

Q1) How is a command being interpreted in linux , the sequence in which it is interpreted ? from left to right ? right to left ?

Shouldn't it be

Code:

test5.sh 2>&1 1>output.txt 
or 
test5.sh 2>&1>output.txt ?

Regards,
Noob