Wednesday, April 8, 2015

ORA-12547 TNS: Lost Contact & telnet: connect: Resource temporarily unavailable

Our database Oracle RAC 11.2.0.3 PSU 6 is running on IBM AIX. 

When we try to use database link we got the error below:

SQL> select * from dual@dblk_prod;
ORA-12547 TNS: Lost Contact

Telnet also got some errors:

/home/root \> telnet localhost 22
Trying...

telnet: connect: Resource temporarily unavailable

With Unix admin we found that the number of CLOSE_WAIT is more than 50k. 

/home/app/oracle > netstat -an|grep CLOSE_WAIT|wc -l
   56564

It turns out that the issue is a bug. And we should apply PSU 7.

Bug 15934834 - osysmond.bin has lot of sockets open in close_wait state (Doc ID 15934834.8)

For workaround we restart CRS and database only in affected node. After that everything is ok.

/home/app/oracle > netstat -an|grep CLOSE_WAIT|wc -l
       3

 /home/root \> telnet localhost 22
Trying...
Connected to loopback.
Escape character is '^]'.

No comments:

Post a Comment