Space Disk "used" In Df Is Nowhere To Be Found With Du

Hello,

I am facing an issue with a filesystem (/dev/sda3); I see space used on it (around 365GB) when I am looking at the host with "df -h" command.

Code:
[root@srv_omega /]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda3             443G  365G   56G  87% /
tmpfs                  95G   56K   95G   1% /dev/shm
/dev/sda1             484M   39M  421M   9% /boot
/dev/sdb1             3.6T  1.3T  2.2T  36% /hadoop/disk1
/dev/sdc1             3.6T  1.3T  2.2T  37% /hadoop/disk2
/dev/sdd1             3.6T  1.3T  2.2T  36% /hadoop/disk3
/dev/sde1             3.6T  1.3T  2.2T  37% /hadoop/disk4
/dev/sdf1             3.6T  1.3T  2.2T  36% /hadoop/disk5
/dev/sdg1             3.6T  1.3T  2.2T  36% /hadoop/disk6
/dev/sdh1             3.6T  1.3T  2.2T  36% /hadoop/disk7
/dev/sdi1             3.6T  1.3T  2.2T  36% /hadoop/disk8
/dev/sdj1             3.6T  1.3T  2.2T  36% /hadoop/disk9
/dev/sdk1             3.6T  1.3T  2.2T  36% /hadoop/disk10
/dev/sdl1             3.6T  1.2T  2.3T  36% /hadoop/disk11
/dev/sdm1             3.6T  1.3T  2.2T  36% /hadoop/disk12
/dev/sdn1             3.6T  1.3T  2.2T  36% /hadoop/disk13
/dev/sdo1             3.6T  1.3T  2.2T  37% /hadoop/disk14
/dev/sdp1             3.6T  1.1T  2.4T  30% /hadoop/disk15
cm_processes           95G  8.2M   95G   1% /var/run/cloudera-scm-agent/process

I have looked if any hidden file might cause the issue, no joy.

Code:
[root@srv_omega /]# pwd
/
[root@srv_omega /]#  ls -lrtha
total 121K
drwxr-xr-x    2 root root 4.0K Jun 28  2011 srv
drwxr-xr-x    2 root root 4.0K Jun 28  2011 mnt
drwxr-xr-x    2 root root 4.0K Jun 28  2011 media
drwxr-xr-x    2 root root 4.0K Dec 20  2012 cgroup
drwx------    2 root root  16K Jun  2  2014 lost+found
drwxr-xr-x    2 root root 4.0K Jun  2  2014 selinux
-rw-r--r--    1 root root    0 Jun  3  2014 .autorelabel
drwxr-xr-x   18 root root 4.0K Jun  5  2014 hadoop
drwxr-xr-x   21 root root 4.0K Jun  5  2014 var
dr-xr-xr-x    9 root root  12K Jun 20  2014 lib64
dr-xr-xr-x    2 root root  12K Jun 21  2014 sbin
dr-xr-xr-x    2 root root 4.0K Jun 21  2014 bin
dr-xr-xr-x    5 root root 1.0K Jun 22  2014 boot
dr-xr-x---    5 root root 4.0K Jun 22  2014 root
drwxr-xr-x    6 root root 4.0K Jun 22  2014 opt
drwxr-xr-x    3 root root 4.0K Dec 10 19:11 home
dr-xr-xr-x   13 root root 4.0K Dec 12 16:18 lib
dr-xr-xr-x 1140 root root    0 Apr 30 15:11 proc
drwxr-xr-x   13 root root    0 Apr 30 15:11 sys
-rw-r--r--    1 root root    0 Apr 30 15:11 .autofsck
drwxr-xr-x    2 root root    0 Apr 30 15:11 misc
drwxr-xr-x    2 root root    0 Apr 30 15:11 net
drwxr-xr-x   15 root root 4.0K Apr 30 15:12 usr
drwxr-xr-x   19 root root 4.6K Apr 30 15:12 dev
dr-xr-xr-x   27 root root 4.0K Apr 30 15:12 ..
dr-xr-xr-x   27 root root 4.0K Apr 30 15:12 .
drwxr-xr-x  122 root root  12K May  4 03:33 etc
drwxrwxrwt   16 root root 4.0K May  7 06:14 tmp

So I try to find where the space is used with a "du -sh" command

Code:
[root@srv_omega /]# pwd
/
[root@srv_omega /]# du -sh *
7.8M    bin
29M     boot
4.0K    cgroup
280K    dev
26M     etc
19T     hadoop
124K    home
144M    lib
26M     lib64
16K     lost+found
4.0K    media
0       misc
4.0K    mnt
0       net
7.9G    opt
du: cannot access `proc/9170/task/27326/fdinfo/538': No such file or directory
du: cannot access `proc/45119/task/45119/fd/4': No such file or directory
du: cannot access `proc/45119/task/45119/fdinfo/4': No such file or directory
du: cannot access `proc/45119/fd/4': No such file or directory
du: cannot access `proc/45119/fdinfo/4': No such file or directory
du: cannot access `proc/45160': No such file or directory
0       proc
3.8M    root
17M     sbin
4.0K    selinux
4.0K    srv
0       sys
3.9M    tmp
2.6G    usr
16G     var

So as far as I understand, only /hadoop is a suitable suspect (as cumulative size of all the other folders on "/" are well below the 365GB)

Code:
[root@srv_omega hadoop]# cd /
[root@srv_omega /]# cd /hadoop
[root@srv_omega hadoop]# ls -lrtha
total 72K
drwxr-xr-x  2 root root 4.0K Jun  5  2014 disk16
drwxr-xr-x 18 root root 4.0K Jun  5  2014 .
drwxr-xr-x  4 root root 4.0K Jun 22  2014 disk1
drwxr-xr-x  4 root root 4.0K Jun 22  2014 disk11
drwxr-xr-x  4 root root 4.0K Jun 22  2014 disk10
drwxr-xr-x  4 root root 4.0K Jun 22  2014 disk13
drwxr-xr-x  4 root root 4.0K Jun 22  2014 disk12
drwxr-xr-x  4 root root 4.0K Jun 22  2014 disk14
drwxr-xr-x  4 root root 4.0K Jun 22  2014 disk2
drwxr-xr-x  4 root root 4.0K Jun 22  2014 disk4
drwxr-xr-x  4 root root 4.0K Jun 22  2014 disk3
drwxr-xr-x  4 root root 4.0K Jun 22  2014 disk6
drwxr-xr-x  4 root root 4.0K Jun 22  2014 disk5
drwxr-xr-x  4 root root 4.0K Jun 22  2014 disk8
drwxr-xr-x  4 root root 4.0K Jun 22  2014 disk7
drwxr-xr-x  4 root root 4.0K Jun 22  2014 disk9
drwxr-xr-x  5 root root 4.0K Nov 19 20:02 disk15
dr-xr-xr-x 27 root root 4.0K Apr 30 15:12 ..

All folders from 1 to 15 are on different filesystems, so the folder disk16 seems to be the only option but there is nothing in it.

Code:
[root@srv_omega hadoop]# cd disk16/
[root@srv_omega disk16]# ls -lrtha
total 8.0K
drwxr-xr-x 18 root root 4.0K Jun  5  2014 ..
drwxr-xr-x  2 root root 4.0K Jun  5  2014 .
[root@srv_omega disk16]#

I just don't get it; no folder seems responsible for the "365Gb"...

Any idea on how I could try to find out where those "365GB" are ?


Similar Content



Sed: Transforming 'ls-laR' Output Into A List With Absolute Paths

Hello, this is my first post
First i would like to thank you all for answering other people questions because I've been able to learn from the forum a lot.

I need your help with something.
I have standard output from 'ls -laR /etc' command which looks like this:
Code:
/etc/X11/xorg.conf.d:
total 4
drwxr-xr-x. 2 root root  29 Apr  1 00:46 .
drwxr-xr-x. 5 root root  54 Apr  1 00:43 ..
-rw-r--r--. 1 root root 232 Apr  1 00:46 00-keyboard.conf

/etc/xdg:
total 12
drwxr-xr-x.  4 root root   36 Apr  1 00:43 .
drwxr-xr-x. 87 root root 8192 Apr 12 13:53 ..
drwxr-xr-x.  2 root root    6 Jun 10  2014 autostart
drwxr-xr-x.  2 root root   17 Apr  7 01:25 systemd

by using sed command:
Code:
sed -e '/./!d' -e '/^total/d' -e '/\.$/d' -e 's/:$/\//' list.txt

I have transformed it to the following form:

Code:
/etc/X11/xorg.conf.d/
-rw-r--r--. 1 root root 232 Apr  1 00:46 00-keyboard.conf
/etc/xdg/
drwxr-xr-x.  2 root root    6 Jun 10  2014 autostart
drwxr-xr-x.  2 root root   17 Apr  7 01:25 systemd

and now I would like to achieve absolute paths at the end of each row

Code:
-rw-r--r--.  1 root root  232 Apr  1 00:46 /etc/X11/xorg.conf.d/00-keyboard.conf
drwxr-xr-x.  2 root root    6 Jun 10  2014 /etc/xdg/autostart
drwxr-xr-x.  2 root root   17 Apr  7 01:25 /etc/xdg/systemd


How do I join(merge) filenames with corresponding absolute path to their parent directory?



I know how to extract filenames using awk and get this:
Code:
00-keyboard.conf

autostart
systemd

but I don't know what to do next. Should I use some hitech sed option or go for loop or try with arrays? Help. Heeeelp

Ln With -t Option, To Create Relative Symlink Not Working

I'm trying to recreate a relative symlink, to link "asymlink" to "somedir/actualfile" in /root/test/ but its creating 2 symlinks instead.

Code:
[root]# ln -t /root/test/ -s somedir/actualfile asymlink
[root]# ll /root/test/
total 4
lrwxrwxrwx 1 root root   18 Feb 16 06:15 actualfile -> somedir/actualfile
lrwxrwxrwx 1 root root    8 Feb 16 06:15 asymlink -> asymlink
drwxr-xr-x 2 root root 4096 Feb 16 06:15 somedir

I can do it with
Code:
cd /root/test
ln -s somedir/actualfile asymlink

but I'm trying to avoid the cd, and also avoid using the full path.

Does anyone know why the -t flag isn't working as expected?

Added 'exit 0' To The Bottom Of .bashrc, Now What?

Centos 6.5

I ran into an interesting problem (on reddit) that I figured I could solve, but I have not been able to. Its simple,.. I added 'exit 0' to /root/.bashrc, and now I am trying to log in via ssh.

Everytime I do, it immediately exits when it runs the .bash_profile, which sources .bashrc, (which is immediate upon 'logging in')

I've tried:

Code:
ssh root@192.168.1.50 -t vim
vim scp://192.168.1.50/.bashrc
vim scp://192.168.1.50/root/.bashrc
ssh root@192.168.1.50 bash --norc
ssh root@192.168.1.50 /bin/bash --norc --noprofile
ssh -T root@192.168.1.50 "mv /root/.bashrc /root/.bashRC"
scp .bashrc root@192.168.1.50:/root/
ssh root@192.168.1.50 /bin/bash --norc --noprofile -vvvvvvvvvv
ssh -vvvvvv root@192.168.1.50 /bin/bash --norc --noprofile
ssh -vvvvvv root@192.168.1.50 /bin/mv /root/.bashrc /root/.bashRC
ssh -t -t root@192.168.1.50 << EOF
mv /root/.bashrc /root/.bashRC
EOF
ssh -t -t root@192.168.1.50 --norc << EOF
echo HELLO > /root/.bashrc
EOF
ssh -tv root@192.168.1.50 rm .bashrc

So,.. I am unable to get back into the system (as root, no other users exist) after adding 'exit 0' to .bashrc

Anyone feel like explaining why all of these failed (aside from saying SSH interactive logins run the .bash_profile/.bashrc files) or, offering a suggestion that works? Seems like if you have the root password, you should be allowed to modify the login process... since... you know... you are root.

Slackware 14.0 - End User Changing Password Failed

Hi all,

I am slackware 14.0 user. Recently I face one problem. As a root i can change one of my normal user's (assuming Peter) password. However, when I login as Peter, and try to change the password, the system does not allow me to renew my password.


root@slackwa /etc# ls -la | grep shadow
-rw-r----- 1 root shadow 9 Nov 29 2013 gshadow
-rw------- 1 root root 0 Jun 20 2013 gshadow-
-rw-r----- 1 root shadow 2272 Apr 12 22:44 shadow
-rw------- 1 root root 2274 Apr 12 21:40 shadow-
root@slackwa /etc# ls -la | grep passwd
-rw-r--r-- 1 root root 2362 Apr 12 12:58 passwd
-rw------- 1 root root 2306 Jan 8 17:38 passwd-
-rw-r--r-- 1 root root 2305 Aug 11 2014 passwd~


peter@slackwa ~$ which passwd
/usr/bin/passwd
peter@slackwa ~$ ls -la /usr/bin/passwd
-rws--x--x 1 root root 68725 Sep 14 2012 /usr/bin/passwd*

peter@slackwa ~$ passwd
Changing password for peter
Old password:
Incorrect password for peter.
The password for peter is unchanged.

One thing I can assure is the password I input is correct. I have another slackware 14 running on Dell R410, but it does not has this problem.


Please kindly advice. Thank you.

SFTP Configuration For Single Directory Per User

Dear Members,

I am trying to configure sftp on my Server (Redhat). But I am facing following issue.

Link used: https://www.linode.com/docs/tools-re...ian-and-ubuntu

These are the parameters added to /etc/ssh/sshd_config file :

Quote:
Match group filetransfer
ChrootDirectory %h
X11Forwarding no
AllowTcpForwarding no
ForceCommand internal-sftp
Permission home Directories are as follows :

Quote:
ls -ld /home/test1/
drwxr-xr-x 7 root root 4096 May 14 09:49 /home/test1/

ls -l /home/test1/
total 12
drwxr-xr-x 2 root root 4096 May 14 09:49 docs
drwxr-xr-x 2 root root 4096 May 13 11:43 nitish
drwxr-xr-x 2 root root 4096 May 14 09:49 public_html

Debug Command : sftp -vvv test1@Ipaddress

Code:
debug1: Next authentication method: password
test1@IP-address's password:
debug3: packet_send2: adding 64 (len 53 padlen 11 extra_pad 64)
debug2: we sent a password packet, wait for reply
debug3: Wrote 144 bytes for a total of 1421
debug1: Authentication succeeded (password).
debug2: fd 4 setting O_NONBLOCK
debug3: fd 5 is O_NONBLOCK
debug1: channel 0: new [client-session]
debug3: ssh_session2_open: channel_new: 0
debug2: channel 0: send open
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug3: Wrote 128 bytes for a total of 1549
debug3: Wrote -1 bytes for a total of 1549
Write failed: Broken pipe
Couldn't read packet: Connection reset by peer

I need the user can't access/see the other directories other than it's own directory

Waiting for your response.

Thank you

Redhat-Keeda

Problem With NFS Sharing Between Two Raspberry Pis

I have two raspberrys, one running OSMC and another running raspbian. The first one has two 1TB hard drives plugged in through a powered usb hub. I want to access to the osmc hard drives from the one running raspbian. They are both in the same local network, the osmc one has the ip 192.168.1.24 and the raspbian one has 192.168.1.28. Both are static ips.

These are the hard drives::
Code:
osmc@osmc:~$ ls /media/ -la
total 36
drwxr-xr-x  4 root root  4096 Mar 31 18:28 .
drwxr-xr-x 23 root root  4096 Mar 15 13:35 ..
drwx------  1 osmc osmc  8192 Mar 30 21:54 ELEMENTS
-rw-r--r--  1 root root   232 Mar  6 13:34 README
drwx------  1 osmc osmc 16384 Mar 30 15:22 TOURO

ELEMENTS and TOURO, two ntfs hard drives that work just fine.

I tried sharing the first one through nfs with the following config (i copied the parameters from a tutorial):
Code:
osmc@osmc:~$ cat /etc/exports
# /etc/exports: the access control list for filesystems which may be exported
#               to NFS clients.  See exports(5).
#
# Example for NFSv2 and NFSv3:
# /srv/homes       hostname1(rw,sync,no_subtree_check) hostname2(ro,sync,no_subtree_check)
#
# Example for NFSv4:
# /srv/nfs4        gss/krb5i(rw,sync,fsid=0,crossmnt,no_subtree_check)
# /srv/nfs4/homes  gss/krb5i(rw,sync,no_subtree_check)
#
/media/ELEMENTS/Pelis/ 192.168.1.0/24(rw,subtree_check,insecure,no_root_squash)
/media/TOURO/Series/ 192.168.1.0/24(rw,subtree_check,insecure,no_root_squash)

and from the pi running raspbian I have confirmed that I can see the drives being shared:
Code:
 /media $ showmount -e 192.168.1.24
Export list for 192.168.1.24:
/media/TOURO/Series/   192.168.1.0/24
/media/ELEMENTS/Pelis/ 192.168.1.0/24

Now, when I try to mount them, all works fine, but when I ls the folders nothing cames back. This is my fstab:
Code:
1cat /etc/fstab
proc            /proc           proc    defaults          0       0
/dev/mmcblk0p1  /boot           vfat    defaults          0       2
/dev/mmcblk0p2  /               ext4    defaults,noatime  0       1
# a swapfile is not a swap partition, so no using swapon|off from here on, use  dphys-swapfile swap[on|off]  for that
UUID=fdff96e6-816c-d001-e05f-96e6816cd001 /media/hdd/ auto defaults,user 0 0 #external hdd

192.168.1.24:/media/ELEMENTS/Pelis /media/pelis nfs rsize=8192,wsize=8192,timeo=14,intr
192.168.1.24:/media/TOURO/Series /media/series nfs rsize=8192,wsize=8192,timeo=14,intr

Code:
apoc@raspbian ~ $ ls /media/pelis/
apoc@raspbian ~ $

The superweird thing is that they are mounted, as they show up if I run "df -h"

Code:
 df -h
S.ficheros                         Tamaņo Usados  Disp Uso% Montado en
rootfs                               7,2G   2,6G  4,4G  37% /
/dev/root                            7,2G   2,6G  4,4G  37% /
devtmpfs                             484M      0  484M   0% /dev
tmpfs                                 98M   380K   98M   1% /run
tmpfs                                5,0M      0  5,0M   0% /run/lock
tmpfs                                195M      0  195M   0% /run/shm
/dev/mmcblk0p1                        56M    15M   42M  26% /boot
/dev/sda1                            219G   4,7G  202G   3% /media/hdd
192.168.1.24:/media/ELEMENTS/Pelis   932G   742G  191G  80% /media/pelis
192.168.1.24:/media/TOURO/Series     932G   813G  120G  88% /media/series

Code:
ls /media/ -la
total 808
drwxr-xr-x  5 root root   4096 abr  2 18:36 .
drwxr-xr-x 22 root root   4096 mar 25 16:14 ..
drwxr-xr-x  5 apoc apoc   4096 abr  2 13:12 hdd
drwx------  1 apoc pi   786432 mar 30 22:03 pelis
drwx------  1 apoc pi    28672 mar 29 16:09 series

Note that the group for the two folders is "pi", but if I umount the drives it becomes "apoc" (my nick and my personal group).
Code:
ls /media/ -la
total 20
drwxr-xr-x  5 root root 4096 abr  2 18:36 .
drwxr-xr-x 22 root root 4096 mar 25 16:14 ..
drwxr-xr-x  5 apoc apoc 4096 abr  2 13:12 hdd
drwxr-xr-x  2 apoc apoc 4096 abr  2 18:36 pelis
drwxr-xr-x  2 apoc apoc 4096 abr  2 18:36 series

Both users ("osmc" in the osmc pi and "apoc" in the raspbian one) have the same uid: 1000.

What am i doing wrong?

Using Cut -d" " To Cut Out The String Before, Not After

Hi Guys,
i have this Code:
uid=0(root) gid=0(root)

i would like to have the word "root" only from the first field, and later from the second field

i use the command
Code:
[root@v1-6 ~]# cat test-uid |cut -d"(" -f1
brms2
uid=0

[root@v1-6 ~]# cat test-uid |cut -d")" -f1
brms2
uid=0(root

it cut the word after the -d, how can i make it to cut the word BEFORE the -d

expected should be:
cut -d"(" -f1
root)

then i can proceed to eliminate the ) later on.
hope this is not confusing you guys

Thanks,

Tree Cmd Is Not Working

Hi All,

Not able to find TREE command in EL6, its working fine in EL5. Need help on this.

Below are the details.



[root@localhost ~]# mkdir -p /college/sports/performer/cricket

[root@localhost ~]# tree /college/
-bash: tree: command not found

[root@localhost ~]# tree college/
-bash: tree: command not found

[root@localhost ~]# which tree
/usr/bin/which: no tree in (/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin: /usr/bin:/root/bin)

[root@localhost ~]# uname -a
Linux localhost.localdomain 2.6.32-71.el6.x86_64 #1 SMP Wed Sep 1 01:33:01 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux

[root@localhost ~]# id
uid=0(root) gid=0(root) groups=0(root),1(bin),2(daemon),3(sys),4(adm),6(disk),10(wheel) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023

Sim Link Question

Does it make any difference where a simlink is performed? Will a simlink be global when done in root or /root? Will the simlink be recognized by all users when done in root or /root?

Thanks.

User Permitted To Run Command,execute Program As Root And After Execution, Exit Root

when a command is typed(i.e. /path/to/the/program), as a normal user, he should be able to run that command , execute that program as ROOT and log out root after the execution is completed.
Only one user should be able to do it.
Hence, I have created a new user vj and added the following command in visudo:
Code:
vj ALL=(ALL) NOPASSWD: /path/to/the/program

Now user vj will be able to typein the command.

What I need is that the program which is been called must run as if it is run by a root user,and when the program is completely executed, exit the root user.

How do I proceed?