Socket recv error 104

socket recv error 104 #4070 Comments OS: Wheezy bitcoind 0.9.1 pre-build libminiupnp 1.6-3 from testing Since the update to 0.9.1 i cant get blocks but a log full of that 104 error. 2014-04-19 07:45:27 ProcessBlock: ACCEPTED 2014-04-19 07:46:46 receive version message: /bitcoinseeder:0.01/: version 60000, blocks=230000, us=[censored]:8333, them=0.0.0.0:0, peer=[2a02:348:5e:5a29::1]:48263 2014-04-19 07:46:58 receive version message: /getaddr.bitnodes.io:0.1/: […]

Содержание

  1. socket recv error 104 #4070
  2. Comments
  3. Footer
  4. [LightGBM] [Fatal] Socket recv error, code: 104 when training with binary objective. #728
  5. Comments
  6. socket recv error Connection reset by peer (104) — BCH #2208
  7. Comments
  8. Footer
  9. socket recv error Connection reset by peer (104) #22
  10. Comments
  11. Footer
  12. cannot connect anymore to peers #90
  13. Comments

socket recv error 104 #4070

OS: Wheezy
bitcoind 0.9.1 pre-build
libminiupnp 1.6-3 from testing

Since the update to 0.9.1 i cant get blocks but a log full of that 104 error.

2014-04-19 07:45:27 ProcessBlock: ACCEPTED
2014-04-19 07:46:46 receive version message: /bitcoinseeder:0.01/: version 60000, blocks=230000, us=[censored]:8333, them=0.0.0.0:0, peer=[2a02:348:5e:5a29::1]:48263
2014-04-19 07:46:58 receive version message: /getaddr.bitnodes.io:0.1/: version 70001, blocks=296603, us=[censored]:8333, them=0.0.0.0:0, peer=[2a01:4f8:131:167::2]:13711
2014-04-19 07:46:58 socket recv error 104
2014-04-19 07:51:15 receive version message: /getaddr.bitnodes.io:0.1/: version 70001, blocks=296603, us=[censored]:8333, them=0.0.0.0:0, peer=[2a01:4f8:131:167::2]:11525
2014-04-19 07:51:16 socket recv error 104
2014-04-19 07:54:08 receive version message: /getaddr.bitnodes.io:0.1/: version 70001, blocks=296603, us=[censored]:8333, them=0.0.0.0:0, peer=[2a01:4f8:131:167::2]:45595
2014-04-19 07:54:08 socket recv error 104
2014-04-19 07:54:40 receive version message: /bitcoinseeder:0.01/: version 60000, blocks=230000, us=[censored]:8333, them=0.0.0.0:0, peer=[2001:470:88ff:2e::1]:35029
2014-04-19 07:57:14 receive version message: /getaddr.bitnodes.io:0.1/: version 70001, blocks=296603, us=[censored]:8333, them=0.0.0.0:0, peer=[2a01:4f8:131:167::2]:28512
2014-04-19 07:57:21 socket recv error 104
2014-04-19 07:58:18 ERROR: AcceptToMemoryPool : nonstandard transaction: dust
2014-04-19 07:58:22 ERROR: AcceptToMemoryPool : nonstandard transaction: dust
2014-04-19 07:58:25 ERROR: AcceptToMemoryPool : nonstandard transaction: dust
2014-04-19 07:58:27 ERROR: AcceptToMemoryPool : nonstandard transaction: dust
2014-04-19 07:58:31 ERROR: AcceptToMemoryPool : nonstandard transaction: dust
2014-04-19 07:58:33 ERROR: AcceptToMemoryPool : nonstandard transaction: dust
2014-04-19 07:58:39 ERROR: AcceptToMemoryPool : nonstandard transaction: dust
2014-04-19 07:58:41 ERROR: AcceptToMemoryPool : nonstandard transaction: dust
2014-04-19 07:58:43 ERROR: AcceptToMemoryPool : nonstandard transaction: dust
2014-04-19 07:58:44 ERROR: AcceptToMemoryPool : nonstandard transaction: dust
2014-04-19 07:58:46 ERROR: AcceptToMemoryPool : nonstandard transaction: dust
2014-04-19 07:58:47 ERROR: AcceptToMemoryPool : nonstandard transaction: dust
2014-04-19 08:02:29 receive version message: /getaddr.bitnodes.io:0.1/: version 70001, blocks=296603, us=[censored]:8333, them=0.0.0.0:0, peer=[2a01:4f8:131:167::2]:40801
2014-04-19 08:02:30 socket recv error 104
2014-04-19 08:03:46 receive version message: /getaddr.bitnodes.io:0.1/: version 70001, blocks=296603, us=[censored]:8333, them=0.0.0.0:0, peer=[2a01:4f8:131:167::2]:58055
2014-04-19 08:03:48 socket recv error 104
2014-04-19 08:04:19 receive version message: /bitcoinseeder:0.01/: version 60000, blocks=230000, us=[censored]:8333, them=0.0.0.0:0, peer=[2a02:348:5e:5a29::1]:43272
2014-04-19 08:06:10 ProcessBlock: ACCEPTED
2014-04-19 08:08:34 receive version message: /getaddr.bitnodes.io:0.1/: version 70001, blocks=296604, us=[censored]:8333, them=0.0.0.0:0, peer=[2a01:4f8:131:167::2]:59327
2014-04-19 08:08:34 socket recv error 104
2014-04-19 08:10:16 keypool reserve 5
2014-04-19 08:10:16 keypool return 5
2014-04-19 08:11:36 receive version message: /getaddr.bitnodes.io:0.1/: version 70001, blocks=296604, us=[censored]:8333, them=0.0.0.0:0, peer=[2a01:4f8:131:167::2]:39861
2014-04-19 08:11:37 socket recv error 104
2014-04-19 08:14:14 receive version message: /getaddr.bitnodes.io:0.1/: version 70001, blocks=296604, us=[censored]:8333, them=0.0.0.0:0, peer=[2a01:4f8:131:167::2]:14525
2014-04-19 08:14:15 socket recv error 104
2014-04-19 08:17:07 ProcessBlock: ACCEPTED
2014-04-19 08:19:18 receive version message: /getaddr.bitnodes.io:0.1/: version 70001, blocks=296605, us=[censored]:8333, them=0.0.0.0:0, peer=[2a01:4f8:131:167::2]:24259
2014-04-19 08:19:20 socket recv error 104
2014-04-19 08:21:57 receive version message: /bitcoinseeder:0.01/: version 60000, blocks=230000, us=[censored]:8333, them=0.0.0.0:0, peer=[2a02:348:5e:5a29::1]:38561
2014-04-19 08:22:19 receive version message: /getaddr.bitnodes.io:0.1/: version 70001, blocks=296605, us=[censored]:8333, them=0.0.0.0:0, peer=[2a01:4f8:131:167::2]:64099
2014-04-19 08:22:20 socket recv error 104

sorry i’m very new to bitcoind but this never happend before.

The text was updated successfully, but these errors were encountered:

Is it possible that your ISP is sending RSTs to close the connections? What sort of connection do you have to the internet? Have you tried connecting via tor?

My ISP use Dual-Stack lite (DS-Lite). All my traffic is tunneled over IPv6 to the Carrier-Grade-NAT-Server of my ISP. This could be the reason. Now i searched the debug log about upnp and found out, it is not working. And GetMyExternalIP() gets the IP of my ISP’s Shared IP all tunneld users get.

So after my first test with onion net it work’s like a charme 🙂

It indeed looks like IPv6 connections are being reset (by the peer, or the ISP?) rougly a second after the version message is received.

© 2023 GitHub, Inc.

You can’t perform that action at this time.

You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.

Источник

[LightGBM] [Fatal] Socket recv error, code: 104 when training with binary objective. #728

Describe the bug
I’ve met a socket error when training with binary objective.

[LightGBM] [Fatal] Socket recv error, code: 104
19/11/07 14:54:06 WARN LightGBMClassifier: LightGBM reached early termination on one worker, stopping training on worker. This message should rarely occur

I’ve tried to train a model during 500 iterations. However, when training 318 iteration, the error has been occurred and stop my training.

It is a similar issue with #569. However objective parameter value is different. In my experiment, the value is «binary».

To Reproduce
Steps to reproduce the behavior, code snippets encouraged

Expected behavior
A clear and concise description of what you expected to happen.

Info (please complete the following information):

  • MMLSpark Version: [e.g. v0.17]
    1.0.0-rc1
  • Spark Version [e.g. 2.4.3]:
    2.3.0
  • Spark Platform [e.g. Databricks]
    AWS EMR

If the bug pertains to a specific feature please tag the appropriate CODEOWNER for better visibility

Additional context
19/11/07 14:41:25 INFO LightGBMClassifier: LightGBM worker calling LGBM_BoosterUpdateOneIter
[LightGBM] [Fatal] Socket recv error, code: 104
19/11/07 14:54:06 WARN LightGBMClassifier: LightGBM reached early termination on one worker, stopping training on worker. This message should rarely occur

The text was updated successfully, but these errors were encountered:

Источник

socket recv error Connection reset by peer (104) — BCH #2208

My BitcoinCash daemon version 0.19.6. Can someone help me. Stuck for few days already

The text was updated successfully, but these errors were encountered:

@solaris-rig Can you show your debug.log (in the data directory) of the BCH daemon? Is it Bitcoin ABC? or something else?

im verifying the current mongodb collection (BCH Chain), cannot connect for the moment. will share the debug.log once i finished current process

the current chain stuck at 530K++ height, I’am trying to get the actual height where the node stop sync

2019-05-26T06:09:48Z received: version (101 bytes) peer=11
2019-05-26T06:09:48Z sending version (114 bytes) peer=11
2019-05-26T06:09:48Z send version message: version 70015, blocks=584220, us=[::]:0, peer=11
2019-05-26T06:09:48Z sending verack (0 bytes) peer=11
2019-05-26T06:09:48Z receive version message: [10.169.6.1:53034] /bitcore:8.0.0/: version 70001, blocks=0, us=[::]:0, peer=11
2019-05-26T06:09:48Z sending alert (168 bytes) peer=11
2019-05-26T06:09:48Z received: verack (0 bytes) peer=11
2019-05-26T06:09:48Z sending ping (8 bytes) peer=11
2019-05-26T06:09:48Z AdvertiseLocal: advertising address pgfkk2abpv2fgxxn.onion:8333
2019-05-26T06:09:48Z sending addr (31 bytes) peer=11
2019-05-26T06:09:48Z initial getheaders (584219) to peer=11 (startheight:0)
2019-05-26T06:09:48Z sending getheaders (1029 bytes) peer=11
2019-05-26T06:09:48Z received: pong (8 bytes) peer=11
2019-05-26T06:09:48Z received: getdata (37 bytes) peer=11
2019-05-26T06:09:48Z received getdata (1 invsz) peer=11
2019-05-26T06:09:48Z received getdata for: block 0000000000000000017195513eeb78388f4d5dd30e886eb9282284127d74f08d peer=11
2019-05-26T06:09:48Z sending block (10281454 bytes) peer=11
2019-05-26T06:09:48Z socket recv error Connection reset by peer (104)
2019-05-26T06:09:48Z disconnecting peer=11
2019-05-26T06:09:48Z Cleared nodestate for peer=11

the message is all the same for block 0000000000000000017195513eeb78388f4d5dd30e886eb9282284127d74f08d

is there anyway to delete certain block and resync again?

last Sync block was at 545957. After that I cant connect to the daemon anymore. Anyone having same problem?

Argh. Nevermind. I updated to version 8.3.3 and everything start working again.

© 2023 GitHub, Inc.

You can’t perform that action at this time.

You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.

Источник

socket recv error Connection reset by peer (104) #22

i use ubuntu, snowgem,but i do not update the block,it tell me
2019-03-21 02:02:45 Loaded 3 addresses from peers.dat 1ms
2019-03-21 02:02:45 dnsseed thread start
2019-03-21 02:02:45 opencon thread start
2019-03-21 02:02:45 addcon thread start
2019-03-21 02:02:45 msghand thread start
2019-03-21 02:02:45 net thread start
2019-03-21 02:02:46 socket recv error Connection reset by peer (104)
2019-03-21 02:02:48 socket recv error Connection reset by peer (104)
2019-03-21 02:02:51 socket recv error Connection reset by peer (104)
2019-03-21 02:02:53 params.size(): 0Loading addresses from DNS seeds (could take a while)
2019-03-21 02:07:20 3 addresses found from DNS seeds
2019-03-21 02:07:20 dnsseed thread exit
2019-03-21 02:13:54 socket recv error Connection reset by peer (104)
2019-03-21 02:14:03 socket recv error Connection reset by peer (104)
2019-03-21 02:14:10 socket recv error Connection reset by peer (104)
help me please

The text was updated successfully, but these errors were encountered:

you should add our nodes:

addnode=dnsseed1.snowgem.org
addnode=dnsseed2.snowgem.org
addnode=dnsseed3.snowgem.org
addnode=explorer.snowgem.org
addnode=pool.snowgem.org

© 2023 GitHub, Inc.

You can’t perform that action at this time.

You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.

Источник

cannot connect anymore to peers #90

since 4 days (4 february) I cannot sync anymore to network

I also try to port forward 8888 with no succes. any idea ?

2018-02-08 19:31:31 socket recv error Connection reset by peer (104)
2018-02-08 19:31:31 connect() to [2001:0:9d38:953c:1ca7:e871:a472:fde9]:8888 failed: Network is unreachable (101)
2018-02-08 19:31:32 socket recv error Connection reset by peer (104)
2018-02-08 19:31:33 connect() to [2001:0:9d38:90d7:c8f:3875:b635:d6a8]:8888 failed: Network is unreachable (101)
2018-02-08 19:31:39 socket recv error Connection reset by peer (104)
2018-02-08 19:31:40 connect() to 46.147.23.0:8888 failed after select(): Connection refused (111)
2018-02-08 19:31:41 socket recv error Connection reset by peer (104)
2018-02-08 19:31:48 socket recv error Connection reset by peer (104)
2018-02-08 19:31:50 socket recv error Connection reset by peer (104)
2018-02-08 19:31:55 connect() to [2001:0:4137:9e76:c60:6ad2:2ac8:4f36]:8888 failed: Network is unreachable (101)
2018-02-08 19:31:56 socket recv error Connection reset by peer (104)
2018-02-08 19:31:57 socket recv error Connection reset by peer (104)
2018-02-08 19:31:58 socket recv error Connection reset by peer (104)
2018-02-08 19:31:59 socket recv error Connection reset by peer (104)
2018-02-08 19:32:05 connect() to [2a01:cb08:5c:300:35e9:622f:2ab0:29f4]:8888 failed: Network is unreachable (101)
2018-02-08 19:32:11 connect() to [2001:0:4137:9e76:4de:3453:b992:c6b9]:8888 failed: Network is unreachable (101)
2018-02-08 19:32:12 connect() to [2001:0:9d38:6abd:c5d:c204:a88b:4f82]:8888 failed: Network is unreachable (101)
2018-02-08 19:32:14 socket recv error Connection reset by peer (104)
2018-02-08 19:32:14 socket recv error Connection reset by peer (104)
2018-02-08 19:32:15 socket recv error Connection reset by peer (104)

The text was updated successfully, but these errors were encountered:

alternatively how can I download current blocks ?
I fixed ipv6 unreachable yet

2018-02-09 18:40:54 socket recv error Connection reset by peer (104)
2018-02-09 18:40:56 socket recv error Connection reset by peer (104)
2018-02-09 18:41:07 connect() to 163.172.70.26:8888 failed after select(): Connection refused (111)
2018-02-09 18:41:14 connect() to 173.212.196.148:8888 failed after select(): Connection refused (111)
2018-02-09 18:41:15 socket recv error Connection reset by peer (104)
2018-02-09 18:41:26 connect() to 163.172.70.26:8888 failed after select(): Connection refused (111)
2018-02-09 18:41:26 connect() to 192.99.3.29:8888 failed after select(): Connection refused (111)
2018-02-09 18:41:32 connect() to 163.172.64.208:8888 failed after select(): Connection refused (111)
2018-02-09 18:41:33 connect() to 37.187.167.145:8888 failed after select(): Connection refused (111)

Hi, I have the same problem and can’t find a solution.
The hushd wallet/node can’t connect. It is few days.
I was experimenting with some parameters and after i used -salvagewallet, then it connected again.
Because I tried -reindex before it was start syncing again.
Then it synced, but I’m not sure if it was 100% synced.
Now it is synced at block 251139 and again — can’t connect to any node.
Tried -reindex and -salvagewallet, but it is not working now.
Also tried to backup wallet.dat. Deleted all the content of /home/user/.hush folder. Generate new config from config generator on myhush website.
Also tried to add a list of nodes from README.md.
But no luck. Still not syncing. It doesn’t matter what ISP I’m using.

This is frustrating. I can’t find no help on the Internet.
Anybody with any help, please?

This is part of debug.log just after I started node. It can run several hours but no connection.

about ipv6 adresses, I couldn’t even ping address (Network is unreachable) so i install package miredo
you will have a new teredo entry in your network interfaces with an ipv6 adress
so now you will also have «Connection refused» using ipv6 (and not Network is unreachable)

telneting on ipv4 adress with 8888 gives connection refused, while testing my external ip adress and husd running functions properly

I am trying reindex, and just see it just start downloadingeverything again 🙁 will see if it stops at same point. in logs i see a lot of
LoadExternalBlockFile: Processing out of order child .

reindexing solved my issue
I use hush swing wallet UI to check 100%
@johnny-burak try also -upgradewallet. just using index solved it, but thought it was related

well that just worked for a day. next day same issue, cannot connect to peers..
any DEV feedback here?
or do we have to download from scratch every day ?

I have similar issues although I set «addnode» in hush.conf. Maybe my network issues?

I ruled out network issues. connection is refused by other servers and database seems corrupted

I suspect a bad closure of daemon (using Ctrl-C instead of hush-cli stop?) or else the swing hush ui is corrupting database

can you guys check in your debug.log if you have such thing:
2018-02-11 18:30:55 ERROR: CheckEquihashSolution(): invalid solution
2018-02-11 18:30:55 ERROR: CheckBlockHeader(): Equihash solution invalid

I also realized that doing :
hush-cli listreceivedbyaddress 0 true
shows only one of my adress with 0 coins, while the one used is not shown

HUSH swing wallet is showing the 2 address

it is definitely a database corruption.. 2 clients have different info. or have diffrerent protocol

I guess i am due for my daily 1.8 gb download reindexing and banning swing wallet for now tocheck that

DEV team, your feedback will be appreciated so we know HUSH is not a dead coin. thanks

@altagir I’ve tried your suggestion -upgradewallet. When I run it then it showed me that can’t load blocks and I have to use -reindex. :-/
So I run in with -reindex oprtion — again.
Now it is running reindexing but still 0 connections.
Tha wallet is unusable like this. I cannot access my coins :-/
I have backup of wallet.dat and exported private key.
But there is no other wallet I can use :-/

@altagir export and backup your private key
when you have loaded wallet.dat from your backup (you can see your coins), then run
hush-cli dumpprivkey «hushaddress» (adress with your coins).
That is extra backup if we can find another working wallet program we can get to our coins by importing private key.
If I understand well there is 1 private key to every hush address, so if you have more addresses with your coins then you have to backup private keys from all your addresses.

to help with your syncing problems. The error LoadExternalBlockFile: Processing out of order child does sound like you CTRL-C’ed or perhaps filled up a harddrive, it’s not a common error, but as you found, a reindex will fix it.

  • the 2 nodes were present in my hush.conf
  • Ctrl-C is normally caught by program, and hush is closed normally
  • I just finished reindexing and then use hush-cli stop, wait a bit, restart daemon, and same problem as before. stuck

so in bried,every time I stop the daemon, it will fail again. and i have to reindex next time

started a fresh .hush while importing my private keys
just used hush-swing-ui to start stop server
let it run all night, restarted it and :

and logs during indexing are

this was a fresh database. lots of HD space, no interruption, and I am not the only one with this issue
starting to be fed up to reindex at every use and download one Gb evey day, is there any other client?

Ok, my testing continue.
After your suggestion -upgradewallet and error
— : Error loading block database. Please restart with -reindex to recover.
I run -reindex yesterday.

During reindex process there was 0 connections at the beginning (i think at least 1 hour). Then I have 4 connections and sync go up to 100% — yes.

Then I go to sleep and exit hush with Ctrl+C as if you don’t run hushd as daemon you can see at terminal [Press Ctrl+C to exit] .
In the debug.log after Ctrl+C hushd is

2018-02-11 22:14:51 connect() to [2a02:c7d:b3a0:1300:9c8a:5c6:67fd:8853]:8888 failed: Network is unreach
able (101)
2018-02-11 22:15:49 tor: Thread interrupt
2018-02-11 22:15:49 torcontrol thread exit
2018-02-11 22:15:49 msghand thread interrupt
2018-02-11 22:15:49 scheduler thread interrupt
2018-02-11 22:15:49 net thread interrupt
2018-02-11 22:15:49 addcon thread interrupt
2018-02-11 22:15:53 opencon thread interrupt
2018-02-11 22:15:53 Shutdown: In progress.
2018-02-11 22:15:53 StopRPC: waiting for async rpc workers to stop
2018-02-11 22:15:53 StopNode()
2018-02-11 22:15:53 Shutdown: done

The same output on debug.log is after I run hush-cli stop.
So I don’t understand why Ctrl+C is bad when it is same as run hush-cli stop.

Today I run hushd. I had 0 connections for almost 30 minutes and I have now 1 wierd connection because I’m at block 255947 but last block at blockachain now is 256471.
Also wallet validated 815 transactions at the start of the program and no more.
So there is definitely something wrong.

I have similar wallets — zend, zclassic and they are working fine.
When I sync the wallet to 100% and I end it with Ctrl+C (as is advised) the next run it connect in few minutes to 8 nodes and syncing the missed blocks.

What is wrong with hushd?
Is there any other wallet for hush?
I really don’t want to reindex every time I run hushd wallet — the syncing take almost 3 hours and it is heavy on CPU load and network data download.

Источник

Источник

OS: Wheezy
bitcoind 0.9.1 pre-build
libminiupnp 1.6-3 from testing

Since the update to 0.9.1 i cant get blocks but a log full of that 104 error.

2014-04-19 07:45:27 ProcessBlock: ACCEPTED
2014-04-19 07:46:46 receive version message: /bitcoinseeder:0.01/: version 60000, blocks=230000, us=[censored]:8333, them=0.0.0.0:0, peer=[2a02:348:5e:5a29::1]:48263
2014-04-19 07:46:58 receive version message: /getaddr.bitnodes.io:0.1/: version 70001, blocks=296603, us=[censored]:8333, them=0.0.0.0:0, peer=[2a01:4f8:131:167::2]:13711
2014-04-19 07:46:58 socket recv error 104
2014-04-19 07:51:15 receive version message: /getaddr.bitnodes.io:0.1/: version 70001, blocks=296603, us=[censored]:8333, them=0.0.0.0:0, peer=[2a01:4f8:131:167::2]:11525
2014-04-19 07:51:16 socket recv error 104
2014-04-19 07:54:08 receive version message: /getaddr.bitnodes.io:0.1/: version 70001, blocks=296603, us=[censored]:8333, them=0.0.0.0:0, peer=[2a01:4f8:131:167::2]:45595
2014-04-19 07:54:08 socket recv error 104
2014-04-19 07:54:40 receive version message: /bitcoinseeder:0.01/: version 60000, blocks=230000, us=[censored]:8333, them=0.0.0.0:0, peer=[2001:470:88ff:2e::1]:35029
2014-04-19 07:57:14 receive version message: /getaddr.bitnodes.io:0.1/: version 70001, blocks=296603, us=[censored]:8333, them=0.0.0.0:0, peer=[2a01:4f8:131:167::2]:28512
2014-04-19 07:57:21 socket recv error 104
2014-04-19 07:58:18 ERROR: AcceptToMemoryPool : nonstandard transaction: dust
2014-04-19 07:58:22 ERROR: AcceptToMemoryPool : nonstandard transaction: dust
2014-04-19 07:58:25 ERROR: AcceptToMemoryPool : nonstandard transaction: dust
2014-04-19 07:58:27 ERROR: AcceptToMemoryPool : nonstandard transaction: dust
2014-04-19 07:58:31 ERROR: AcceptToMemoryPool : nonstandard transaction: dust
2014-04-19 07:58:33 ERROR: AcceptToMemoryPool : nonstandard transaction: dust
2014-04-19 07:58:39 ERROR: AcceptToMemoryPool : nonstandard transaction: dust
2014-04-19 07:58:41 ERROR: AcceptToMemoryPool : nonstandard transaction: dust
2014-04-19 07:58:43 ERROR: AcceptToMemoryPool : nonstandard transaction: dust
2014-04-19 07:58:44 ERROR: AcceptToMemoryPool : nonstandard transaction: dust
2014-04-19 07:58:46 ERROR: AcceptToMemoryPool : nonstandard transaction: dust
2014-04-19 07:58:47 ERROR: AcceptToMemoryPool : nonstandard transaction: dust
2014-04-19 08:02:29 receive version message: /getaddr.bitnodes.io:0.1/: version 70001, blocks=296603, us=[censored]:8333, them=0.0.0.0:0, peer=[2a01:4f8:131:167::2]:40801
2014-04-19 08:02:30 socket recv error 104
2014-04-19 08:03:46 receive version message: /getaddr.bitnodes.io:0.1/: version 70001, blocks=296603, us=[censored]:8333, them=0.0.0.0:0, peer=[2a01:4f8:131:167::2]:58055
2014-04-19 08:03:48 socket recv error 104
2014-04-19 08:04:19 receive version message: /bitcoinseeder:0.01/: version 60000, blocks=230000, us=[censored]:8333, them=0.0.0.0:0, peer=[2a02:348:5e:5a29::1]:43272
2014-04-19 08:06:10 ProcessBlock: ACCEPTED
2014-04-19 08:08:34 receive version message: /getaddr.bitnodes.io:0.1/: version 70001, blocks=296604, us=[censored]:8333, them=0.0.0.0:0, peer=[2a01:4f8:131:167::2]:59327
2014-04-19 08:08:34 socket recv error 104
2014-04-19 08:10:16 keypool reserve 5
2014-04-19 08:10:16 keypool return 5
2014-04-19 08:11:36 receive version message: /getaddr.bitnodes.io:0.1/: version 70001, blocks=296604, us=[censored]:8333, them=0.0.0.0:0, peer=[2a01:4f8:131:167::2]:39861
2014-04-19 08:11:37 socket recv error 104
2014-04-19 08:14:14 receive version message: /getaddr.bitnodes.io:0.1/: version 70001, blocks=296604, us=[censored]:8333, them=0.0.0.0:0, peer=[2a01:4f8:131:167::2]:14525
2014-04-19 08:14:15 socket recv error 104
2014-04-19 08:17:07 ProcessBlock: ACCEPTED
2014-04-19 08:19:18 receive version message: /getaddr.bitnodes.io:0.1/: version 70001, blocks=296605, us=[censored]:8333, them=0.0.0.0:0, peer=[2a01:4f8:131:167::2]:24259
2014-04-19 08:19:20 socket recv error 104
2014-04-19 08:21:57 receive version message: /bitcoinseeder:0.01/: version 60000, blocks=230000, us=[censored]:8333, them=0.0.0.0:0, peer=[2a02:348:5e:5a29::1]:38561
2014-04-19 08:22:19 receive version message: /getaddr.bitnodes.io:0.1/: version 70001, blocks=296605, us=[censored]:8333, them=0.0.0.0:0, peer=[2a01:4f8:131:167::2]:64099
2014-04-19 08:22:20 socket recv error 104

sorry i’m very new to bitcoind but this never happend before.

Источник

I have a code where there are 13 clients that have to connect to the server. Then the server does some counting on the data given by the client. After that the roles turns around — the server becomes a client and clients become servers to receive the data.
The thing is that when trying to do the first connection, that is when the 13 clients try to connect to the server I keep getting this error: [Errno 104] Connection reset by peer. I tried some workarounds for example trying to connect 5 times in a second interval but nothing works.

Here my code:

server.py

import socket, pickle, numpy as np
import struct
import math


while 1:
    HOST = ''
    PORT = 50007
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.bind((HOST, PORT))
    s.listen(13)

adresses = []
ports = []

i = 0
print("receiving...")
while i < 13:
    i += 1    
    #wait to accept a connection - blocking call
    conn, addr = s.accept()
    print ('Connected with ', addr)
    adresses.append(addr[0])
    buf = b''
    while len(buf) < 4:
        buf += conn.recv(4 - len(buf))
    length = struct.unpack('>I', buf)[0]
    data = b''
    l = length

    while l > 0:

        d = conn.recv(l)
        l -= len(d)
        data += d

    if not data: break

    M = np.loads(data)

    if i == 1:
        L = M[0]
    else:
        L += M[0]
    ports.append(M[1])
    conn.close() 
s.close()


L /= 993040

packet = pickle.dumps(L)
length = struct.pack('>I', len(packet))
packet = length + packet

print("sending...")
for kl, addr in enumerate(adresses):
    HOST = addr
    PORT = 50007 + ports[kl]
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.connect((HOST, PORT))
    s.sendall(packet)
    s.close()

client.py

def connection(centers, kl):

    HOST = "192.168.143.XX"
    PORT = 50007
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.settimeout(3600)
    try:
        s.connect((HOST, PORT)) # HERE IS AN ERROR
        s.settimeout(None)
        packet = pickle.dumps([centers, kl]) ## ???
        length = struct.pack('>I', len(packet))
        packet = length + packet
        s.sendall(packet) # OR HERE IS AN ERROR
        s.close()
    except Exception as e:
        print(e)
        print('error ', kl)
        s.close()
        return np.zeros(centers.shape)    


    HOST = ''
    PORT = 50007 + kl
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.bind((HOST, PORT))

    s.listen(2)

    i = 0
    while i < 1:
        #wait to accept a connection - blocking call
        conn, addr = s.accept()
        i += 1
        print ('Connected with ', addr)
        buf = b''
        while len(buf) < 4:
            buf += conn.recv(4 - len(buf))

        length = struct.unpack('>I', buf)[0]
        data = b''
        l = length
        while l > 0:
            d = conn.recv(l)
            l -= len(d)
            data += d
        if not data: break

        new_centers = np.loads(data)
        conn.close()

    s.close()
    return new_centers

aa = 0
for k in range(99):
    print(k)
    centers = some_function(centers)
    time.sleep(60)
    centers1 = connection(centers, i)
    aa = 0
    while not (centers1.any()) and aa < 5:
        time.sleep(1)
        centers1 = connection(centers, i)
        aa += 1
    centers = centers1

The thing is all of the 13 clients HAVE TO connect to the server or it won’t proceed to the next iteration.
I’m using Python 3.4.
Please help.

Update:

I have added threads but the error remains:

[Errno 104] Connection reset by peer

server.py

import socket, pickle, numpy as np
import struct
import math
from multiprocessing.pool import ThreadPool

def clientthread(conn, L):
    buf = b''
    while len(buf) < 4:
        buf += conn.recv(4 - len(buf))
    length = struct.unpack('>I', buf)[0]
    data = b''
    l = length
    while l > 0:
        d = conn.recv(l)
        l -= len(d)
        data += d
    M = np.loads(data)


    return(M)


j = 0
while 1:
    HOST = ''
    PORT = 50007
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.bind((HOST, PORT))
    s.listen(2)
    #print('0')

    adresses = []
    ports = []

    i = 0
    print("receiving...")
    while i < 13:
        i += 1    
        #wait to accept a connection - blocking call
        conn, addr = s.accept()
        print ('Connected with ', addr)
        adresses.append(addr[0])
        pool = ThreadPool(processes=13)
        async_result = pool.apply_async(clientthread, (conn, i,))
        M = async_result.get()  
        conn.close() 

        if i == 1:
            L = M[0]
        else:
            L += M[0]
        ports.append(M[1])
    s.close()


    L /= 993040

    packet = pickle.dumps(L)
    length = struct.pack('>I', len(packet))
    packet = length + packet

    for kl, addr in enumerate(adresses):
        HOST = addr
        PORT = 50007 + ports[kl]
        s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        s.connect((HOST, PORT))
        s.sendall(packet)
        s.close()

Источник

So, if I do some benchmarking with apache benchmark (ab), and I use large numbers of requests. Then sometimes in the middle of a test I get this error.

I don’t even know what it means. So how can I fix it? Or is it just something that will happen if the server gets too many hits anyway? The problem is, if I run 10,000 hits, it’ll all run perfectly. If I run it again, it’ll get to 4000 and get the error:

apr_socket_recv: Connection reset by peer (104)

A little about my setup:
I have nginx taking static requests and processing dynamic ones to apache. The file in question is served from cache by nginx, so I guess it’s probably got to do with how nginx is handling the requests?

Ideas?

asked May 30, 2010 at 0:59

Matthew's user avatar

The error means that the other end (webserver) suddenly disconnected in the middle of the session.
have a look at the apache or nginx error logs to see if there is anything suspicious there.

answered May 30, 2010 at 7:43

Aleksandar Ivanisevic's user avatar

It means that server is heavly loaded with the request i.e, all the threads are busy serving the request.
Solution : either increase the maxThread attribute count for connector in server.xml file or increase acceptCount attribute value.

acceptcount : The maximum queue length for incoming connection requests when all possible request processing threads are in use. Any requests received when the queue is full will be refused.

answered Mar 13, 2014 at 7:21

Kushal Bafna's user avatar

Besides the answers here, I have read a lot of other ones:

  • Replace localhost by 127.0.0.1
  • Update the apache version (I have ApacheBench, Version 2.3 <$Revision: 1807734 $>)
  • Add -r (Then I get apr_pollset_poll: The timeout specified has expired (70007))

None of them helped.

I thought about switching to wrk after seeing similar struggles.

Finding the problem

The problem seems to be related to the amount of ephermal ports. I tried to set it from 50000 to 25000 as this is the port range. Still no luck. Then I got the impression it is related to TIME_WAIT and this blog post. I think I could confirm that:

$ netstat -nat | awk '{print $6}' | sort | uniq -c | sort -n

    1 CLOSE_WAIT
    1 established)
    1 Foreign
    4 LISTEN
    8 SYN_SENT
   62 SYN_RECV
  351 ESTABLISHED
13916 TIME_WAIT

What I tried

I didn’t fix it so far :-/

According to sudo sysctl -a | grep net.ipv4.tcp, I have:

net.ipv4.tcp_tw_reuse = 0    # No luck setting only that to 1
net.ipv4.tcp_max_tw_buckets = 32768
net.ipv4.tcp_fin_timeout = 60  # Setting it to 5 didn't help either

answered Jul 11, 2018 at 21:24

Martin Thoma's user avatar

This issue is caused by the system. if give a high concurrency request to the system. OS kernel will trigger SYN flood protection. So the system will reset the link.
you can modify the OS config in the file.

#vi /etc/sysctl.conf
net.ipv4.tcp_syncookies = 0 # set value is 0
#sysctl -p # read config from the config file.

you can try it.

usually the attribute net.ipv4.tcp_syncookies was used to protect OS to avoid the huge request attack. But if you want to use this OS to do some Load test or performance test, you should close this feature.

Jenny D's user avatar

Jenny D

27.5k21 gold badges74 silver badges112 bronze badges

answered Nov 30, 2017 at 22:46

moneyfly's user avatar

I had same problem and my server version was:

Server Version: Apache/2.4.6 (CentOS) OpenSSL/1.0.1e-fips mod_fcgid/2.3.9 PHP/5.6.5 mod_perl/2.0.9dev Perl/v5.16.3

i removed unneccessary modules and problem is gone:

Server Version: Apache/2.4.6 (CentOS) OpenSSL/1.0.1e-fips

So one of mod_fcgid, mod_php or mod_perl is causing problem. You can try to disable those if you are not using.

(Side note;
If you are using opcache,disable fast_shutdown too. It was causing problem too:
opcache.fast_shutdown=0 )

answered Feb 23, 2015 at 6:30

Ünsal Korkmaz's user avatar

0

Источник

Здравствуйте.
Ubuntu 16, FastCGI (Nginx + PHP-FPM), PHP 7.2.

Со вчерашнего дня сайт периодически выдает 502 ошибку, в лог пишет:

2018/09/02 08:41:18 [error] 7016#7016: *7584 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: xx.xxx.xxx.xxx, server: site.com, request: "GET /page/url/ HTTP/1.1", upstream: "fastcgi://unix:/run/php/php7.2-fpm.sock:", host: "site.com"

Мой конфиг:

user www-data;
worker_processes auto;
worker_rlimit_nofile 65000;
timer_resolution 100ms;
worker_priority -5;

error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;

events {
	worker_connections 4096;
	multi_accept on;
	use epoll;
}

http {
	include      /etc/nginx/mime.types;
	default_type application/octet-stream;

	log_format main '$remote_addr - $remote_user [$time_local] "$request" '
					'$status $body_bytes_sent "$http_referer" '
					'"$http_user_agent" "$http_x_forwarded_for"';

	access_log /var/log/nginx/access.log main;

	sendfile on;
	keepalive_timeout 65;

	reset_timedout_connection on;

	include /etc/nginx/conf.d/*.conf;
	include /etc/nginx/vhosts/*/*.conf;

	client_body_buffer_size 8m;
	client_max_body_size    128m;

	open_file_cache max=200000 inactive=20s; 
	open_file_cache_valid 30s; 
	open_file_cache_min_uses 2;
	open_file_cache_errors on;

	server {
		server_name      localhost;
		disable_symlinks if_not_owner;

		include /etc/nginx/vhosts-includes/*.conf;

		location @fallback {
			error_log /dev/null crit;

			proxy_pass http://127.0.0.1:8080;
			proxy_redirect http://127.0.0.1:8080 /;
			proxy_set_header Host $host;
			proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
			proxy_set_header X-Forwarded-Proto $scheme;

			access_log off ;
		}

		listen 80;
		listen [::]:80;
	}
}

И второй конфиг:

server {
	server_name site.com www.site.com;
	charset UTF-8;
	index index.php;
	disable_symlinks if_not_owner from=$root_path;
	include /etc/nginx/vhosts-includes/*.conf;
	include /etc/nginx/vhosts-resources/site.com/*.conf;
	error_log /var/www/httpd-logs/site.com.error.log crit;
	ssi on;
	return 301 https://$host:443$request_uri;
	set $root_path /var/www/www-root/data/www/site.com;
	root $root_path;
	gzip on;
	gzip_comp_level 5;
	gzip_disable "msie6";
	gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript application/javascript;
	error_log /var/www/httpd-logs/site.com.error.log notice;
	location ~* ^.+.(jpg|jpeg|gif|png|ico|css|js|ico)$ {
		expires 7d;
	}
	location / {
		try_files $uri /index.php?do=$uri;
		location ~ [^/].ph(pd*|tml)$ {
			try_files /does_not_exists @php;
		}
	}
	location @php {
		fastcgi_index index.php;
		fastcgi_param PHP_ADMIN_VALUE "sendmail_path = /usr/sbin/sendmail -t -i -f znleha@yandex.ru";
		fastcgi_pass unix:/run/php/php7.2-fpm.sock;
		fastcgi_split_path_info ^((?U).+.ph(?:pd*|tml))(/?.+)$;
		try_files $uri =404;

		fastcgi_buffers 4 512k;
		fastcgi_busy_buffers_size 512k;
		fastcgi_temp_file_write_size 512k;

		include fastcgi_params;
	}
	access_log off;
	listen 77.222.63.224:80 default_server;
	listen [2a02:408:7722:54:77:222:63:224]:80 default_server;
}
server {
	server_name site.com www.site.com;
	ssl_certificate "/var/www/httpd-cert/www-root/site.com_le1.crtca";
	ssl_certificate_key "/var/www/httpd-cert/www-root/site.com_le1.key";
	ssl_ciphers EECDH:+AES256:-3DES:RSA+AES:!NULL:!RC4;
	ssl_prefer_server_ciphers on;
	ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
	add_header Strict-Transport-Security "max-age=31536000;";
	ssl_dhparam /etc/ssl/certs/dhparam4096.pem;
	charset UTF-8;
	index index.php;
	disable_symlinks if_not_owner from=$root_path;
	include /etc/nginx/vhosts-includes/*.conf;
	include /etc/nginx/vhosts-resources/site.com/*.conf;
	error_log /var/www/httpd-logs/site.com.error.log crit;
	ssi on;
	set $root_path /var/www/www-root/data/www/site.com;
	root $root_path;
	gzip on;
	gzip_comp_level 5;
	gzip_disable "msie6";
	gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript application/javascript;
	error_log /var/www/httpd-logs/site.com.error.log notice;
	listen [2a02:408:7722:54:77:222:63:224]:443 ssl default_server;
	listen 77.222.63.224:443 ssl default_server;
	location ~* ^.+.(jpg|jpeg|gif|png|ico|css|js|ico)$ {
		expires 7d;
	}
	location / {
		try_files $uri /index.php?do=$uri;
		location ~ [^/].ph(pd*|tml)$ {
			try_files /does_not_exists @php;
		}
	}
	location @php {
		fastcgi_index index.php;
		fastcgi_param PHP_ADMIN_VALUE "sendmail_path = /usr/sbin/sendmail -t -i -f znleha@yandex.ru";
		fastcgi_pass unix:/run/php/php7.2-fpm.sock;
		fastcgi_split_path_info ^((?U).+.ph(?:pd*|tml))(/?.+)$;
		try_files $uri =404;

		fastcgi_buffers 4 512k;
		fastcgi_busy_buffers_size 512k;
		fastcgi_temp_file_write_size 512k;

		include fastcgi_params;
	}
	access_log off;
}

Ошибка может появиться в любой момент (может и не появиться)… От куда ноги растут, не пойму…
Подскажите, куда копать?
Спасибо.

Источник

SW Test Automation Framework

  • Summary

  • Files

  • Reviews

  • Support

  • Mailing Lists

  • Tickets ▾

    • Bugs
    • Feature Requests
    • Patches
    • Support Requests

  • News

  • Discussion

  • Code

Menu
▾
▴

Error reading from socket: recv() RC=104: 22


Created:

2010-03-17

Updated:

2013-06-12

  • Alan Massie

    Hi All,

    Have been experiencing a new issue with STAF lately, I have been unable to completely debug it during its non-working state as it has been happening on a system that I can not leave in that state long enough to debug further. As of this moment the only information I have is below, but would welcome any comments or suggestions on this particular issue.

    evsadm@testmgr02:~$ staf local ping ping
    Error registering with STAF, RC: 22

    evsadm@testmgr02:~$ staf local ping ping
    Error submitting request, RC: 22
    Additional info

    STAFConnectionReadUInt: Error reading from socket: recv() RC=104: 22

    root@testmgr02:~# staf local var list
    Response

    STAF/Config/BootDrive           : /
    STAF/Config/CodePage            : ISO-8859-1
    STAF/Config/ConfigFile          : /usr/local/staf/bin/STAF.cfg
    STAF/Config/DefaultAuthenticator: none
    STAF/Config/DefaultInterface    : tcp
    STAF/Config/InstanceName        : STAF
    STAF/Config/Machine             : testmgr02
    STAF/Config/MachineNickname     : testmgr02
    STAF/Config/Mem/Physical/Bytes  : 8260980736
    STAF/Config/Mem/Physical/KB     : 8067364
    STAF/Config/Mem/Physical/MB     : 7878
    STAF/Config/OS/MajorVersion     : 2.6.28-17-server
    STAF/Config/OS/MinorVersion     : #58-Ubuntu SMP Tue Dec 1 22:13:36 UTC 2009
    STAF/Config/OS/Name             : Linux
    STAF/Config/OS/Revision         : x86_64
    STAF/Config/Processor/NumAvail  : 4
    STAF/Config/Sep/Command         : ;
    STAF/Config/Sep/File            : /
    STAF/Config/Sep/Line            :

    STAF/Config/Sep/Path            : :
    STAF/Config/STAFRoot            : /usr/local/staf
    STAF/Config/StartupTime         : 20100317-12:33:06
    STAF/DataDir                    : /usr/local/staf/data/STAF
    STAF/Env/_                      : /usr/local/staf/bin/STAFProc
    STAF/Env/CLASSPATH              : /usr/local/staf/lib/JSTAF.jar:/usr/local/staf/samples/demo/STAFDemo.jar::/usr/local/staf/services/stax/STAX.jar:/usr/local/staf/lib/JSTAF.jar
    STAF/Env/HOME                   : /home/evsadm
    STAF/Env/LANG                   : en_US
    STAF/Env/LD_LIBRARY_PATH        : /usr/local/staf/lib::/usr/local/staf/lib
    STAF/Env/LOGNAME                : evsadm
    STAF/Env/MAIL                   : /var/mail/evsadm
    STAF/Env/PATH                   : /usr/local/staf/bin:/home/evsadm/bin:/usr/local/bin:/usr/bin:/bin:/usr/games:/usr/local/staf/bin
    STAF/Env/PWD                    : /home/evsadm
    STAF/Env/SHELL                  : /bin/bash
    STAF/Env/SHLVL                  : 1
    STAF/Env/STAF_INSTANCE_NAME     : STAF
    STAF/Env/STAFCONVDIR            : /usr/local/staf/codepage
    STAF/Env/TERM                   : xterm
    STAF/Env/USER                   : evsadm
    STAF/Env/XDG_SESSION_COOKIE     : 5e28b0fe2cda5d20d19a7ebf4b50a4ea-1268854385.277630-21708366
    STAF/Version                    : 3.3.4

    root@testmgr02:~# STAF local PROCESS START SHELL COMMAND «java -version» RETURNSTDOUT STDERRTOSTDOUT WAIT
    Response

    {
      Return Code: 0
      Key        : <None>
      Files      : [
        {
          Return Code: 0
          Data       : java version «1.6.0_0»
    OpenJDK Runtime Environment (IcedTea6 1.4.1) (6b14-1.4.1-0ubuntu12)
    OpenJDK 64-Bit Server VM (build 14.0-b08, mixed mode)

        }
      ]
    }

  • Sharon Lucas

    The socket error number in the error message provided by STAF is 104 after calling recv().  if you google for this socket recv() errno you’ll find it means ECONNRESET (Connection reset by peer), which is caused by the remote end dropping the connection before you try to send data. This could be caused by anything from a firewall to a network problem, etc. but since you’re getting it submitting a STAF service request to the local connection provider, it wouldn’t be caused by a firewall or network problem.  Perhaps the STAF local IPC connection provider has «died».

    Do all STAF requests submitted to the «local» connection provider fail when it is in this state?

    Can another machine submit a STAF PING request to this machine using its host name or IP address?

    Is there any additional information provided in STAFProc’s stdout/stderr?  That is, when you start STAFProc, make sure that you are redirect its output to a file like startSTAFProc.sh does and then check that file to see if there is any additional error information.

  • Alan Massie

    STAF is started with output redirected like this.

    start() {
        echo «starting staf, waiting for 20 seconds to issue staf commands»
        if ; then
               shift;
               su — evsadm -c «. /usr/local/staf/STAFEnv.sh && /usr/local/staf/bin/STAFProc > /tmp/stafproc.out 2>&1 &»
        else
               su — evsadm -c «/usr/local/staf/bin/STAFProc > /tmp/stafproc.out 2>&1 &»

    Machine          : testmgr02
    Machine nickname : testmgr02
    Startup time     : 20100317-12:33:06

    STAFProc version 3.3.4 initialized

    Since the problem, STAF has been restarted, but I am also looking into the possibility of the error being due to the
    amount of sockets being left open by java process ? Again as we fixed the problem temporarily it will be difficult to provide more
    details, but when it happens again, which im sure it will, I will gather more information and update this post.

    Thanks.

  • Alan Massie

    Forgot to add that yes another machine can submit a STAF PING request and receives a PONG.
    Also that when running further STAF requests while in this state it has three different returns;

    evsadm@testmgr02:~$ staf local ping ping
    Error registering with STAF, RC: 22

    evsadm@testmgr02:~$ staf local ping ping
    Error submitting request, RC: 22
    Additional info STAFConnectionReadUInt: Error reading from socket: recv() RC=104: 22

    evsadm@testmgr02:~# staf local ping ping
    evsadm@testmgr02:~#

    The last one being no response at all, which is puzzling.

  • Sharon Lucas

    So, it is only requests submitted to the local machine using the local interface (not the tcp or ssl  interface) that are failing, usually with RC 22.  Note that the STAF LocalIPC Connection Provider on Unix machines (aka «local» interface) uses sockets for inter-process communication (IPC).  That’s what the STAF socket files in the /tmp are used for.  For example:

    # ls -l /tmp
    srwxrwxrwx   1 root root    0 Mar 10 09:08 STAFIPC_STAF
    srwxrwxrwx   1 root root    0 Mar 10 09:08 STAFIPC_STAFJSTAF_STAFJVM1

    So, I think you could be on the right track looking into the possibility of the error being due to the amount of sockets being left open by some other process.

  • Sharon Lucas

    You can use lsof to list the open files for a process (such as the java process you mentioned) to see if files are being left open.  For example:

    lsof -p <PID of process>

    Are you running the STAX service on this machine?  If so, also check the JVM log for the STAX service for error messages (e.g.  /usr/local/staf/data/STAF/lang/java/jvm/<JVMName>/JVMLog.1  (where <JVMName> will either be STAFJVM1 or whatever JVM name you specified when registering the STAX service in your STAF.cfg file)

  • Alan Massie

    Here is a LSOF top and other outputs from our hang over the weekend, did not manage to get a lsof -p as I was not the one collecting the data. We are running a slightly heavier load of tests just now, but I am at a loss of where in STAF i can modify settings to have JAVA close files or time them out after an amount of time.

    COMMAND     PID       USER   FD      TYPE             DEVICE     SIZE       NODE NAME
    init          1       root  cwd   unknown                                        /proc/1/cwd (readlink: Permission denied)
    init          1       root  rtd   unknown                                        /proc/1/root (readlink: Permission denied)
    init          1       root  txt   unknown                                        /proc/1/exe (readlink: Permission denied)
    init          1       root NOFD                                                  /proc/1/fd (opendir: Permission denied)
    kthreadd      2       root  cwd   unknown                                        /proc/2/cwd (readlink: Permission denied)
    kthreadd      2       root  rtd   unknown                                        /proc/2/root (readlink: Permission denied)
    kthreadd      2       root  txt   unknown                                        /proc/2/exe (readlink: Permission denied)
    kthreadd      2       root NOFD                                                  /proc/2/fd (opendir: Permission denied)
    migration     3       root  cwd   unknown                                        /proc/3/cwd (readlink: Permission denied)
    migration     3       root  rtd   unknown                                        /proc/3/root (readlink: Permission denied)
    migration     3       root  txt   unknown                                        /proc/3/exe (readlink: Permission denied)
    migration     3       root NOFD                                                  /proc/3/fd (opendir: Permission denied)
    ksoftirqd     4       root  cwd   unknown                                        /proc/4/cwd (readlink: Permission denied)
    ksoftirqd     4       root  rtd   unknown                                        /proc/4/root (readlink: Permission denied)
    ksoftirqd     4       root  txt   unknown                                        /proc/4/exe (readlink: Permission denied)
    ksoftirqd     4       root NOFD                                                  /proc/4/fd (opendir: Permission denied)
    watchdog/     5       root  cwd   unknown                                        /proc/5/cwd (readlink: Permission denied)
    watchdog/     5       root  rtd   unknown                                        /proc/5/root (readlink: Permission denied)
    watchdog/     5       root  txt   unknown                                        /proc/5/exe (readlink: Permission denied)
    watchdog/     5       root NOFD                                                  /proc/5/fd (opendir: Permission denied)
    migration     6       root  cwd   unknown                                        /proc/6/cwd (readlink: Permission denied)
    migration     6       root  rtd   unknown                                        /proc/6/root (readlink: Permission denied)
    migration     6       root  txt   unknown                                        /proc/6/exe (readlink: Permission denied)
    migration     6       root NOFD                                                  /proc/6/fd (opendir: Permission denied)
    ksoftirqd     7       root  cwd   unknown                                        /proc/7/cwd (readlink: Permission denied)
    ksoftirqd     7       root  rtd   unknown                                        /proc/7/root (readlink: Permission denied)
    ksoftirqd     7       root  txt   unknown                                        /proc/7/exe (readlink: Permission denied)
    ksoftirqd     7       root NOFD                                                  /proc/7/fd (opendir: Permission denied)
    watchdog/     8       root  cwd   unknown                                        /proc/8/cwd (readlink: Permission denied)
    watchdog/     8       root  rtd   unknown                                        /proc/8/root (readlink: Permission denied)
    watchdog/     8       root  txt   unknown                                        /proc/8/exe (readlink: Permission denied)
    watchdog/     8       root NOFD                                                  /proc/8/fd (opendir: Permission denied)
    migration     9       root  cwd   unknown                                        /proc/9/cwd (readlink: Permission denied)
    migration     9       root  rtd   unknown                                        /proc/9/root (readlink: Permission denied)
    migration     9       root  txt   unknown                                        /proc/9/exe (readlink: Permission denied)
    migration     9       root NOFD                                                  /proc/9/fd (opendir: Permission denied)
    ksoftirqd    10       root  cwd   unknown                                        /proc/10/cwd (readlink: Permission denied)
    ksoftirqd    10       root  rtd   unknown                                        /proc/10/root (readlink: Permission denied)
    ksoftirqd    10       root  txt   unknown                                        /proc/10/exe (readlink: Permission denied)
    ksoftirqd    10       root NOFD                                                  /proc/10/fd (opendir: Permission denied)
    watchdog/    11       root  cwd   unknown                                        /proc/11/cwd (readlink: Permission denied)
    watchdog/    11       root  rtd   unknown                                        /proc/11/root (readlink: Permission denied)
    watchdog/    11       root  txt   unknown                                        /proc/11/exe (readlink: Permission denied)
    watchdog/    11       root NOFD                                                  /proc/11/fd (opendir: Permission denied)
    migration    12       root  cwd   unknown                                        /proc/12/cwd (readlink: Permission denied)
    migration    12       root  rtd   unknown                                        /proc/12/root (readlink: Permission denied)
    migration    12       root  txt   unknown                                        /proc/12/exe (readlink: Permission denied)
    migration    12       root NOFD                                                  /proc/12/fd (opendir: Permission denied)
    ksoftirqd    13       root  cwd   unknown                                        /proc/13/cwd (readlink: Permission denied)
    ksoftirqd    13       root  rtd   unknown                                        /proc/13/root (readlink: Permission denied)
    ksoftirqd    13       root  txt   unknown                                        /proc/13/exe (readlink: Permission denied)
    ksoftirqd    13       root NOFD                                                  /proc/13/fd (opendir: Permission denied)
    watchdog/    14       root  cwd   unknown                                        /proc/14/cwd (readlink: Permission denied)
    watchdog/    14       root  rtd   unknown                                        /proc/14/root (readlink: Permission denied)
    watchdog/    14       root  txt   unknown                                        /proc/14/exe (readlink: Permission denied)
    watchdog/    14       root NOFD                                                  /proc/14/fd (opendir: Permission denied)
    events/0     15       root  cwd   unknown                                        /proc/15/cwd (readlink: Permission denied)
    events/0     15       root  rtd   unknown                                        /proc/15/root (readlink: Permission denied)
    events/0     15       root  txt   unknown                                        /proc/15/exe (readlink: Permission denied)
    events/0     15       root NOFD                                                  /proc/15/fd (opendir: Permission denied)
    events/1     16       root  cwd   unknown                                        /proc/16/cwd (readlink: Permission denied)
    events/1     16       root  rtd   unknown                                        /proc/16/root (readlink: Permission denied)
    events/1     16       root  txt   unknown                                        /proc/16/exe (readlink: Permission denied)
    events/1     16       root NOFD                                                  /proc/16/fd (opendir: Permission denied)
    events/2     17       root  cwd   unknown                                        /proc/17/cwd (readlink: Permission denied)
    events/2     17       root  rtd   unknown                                        /proc/17/root (readlink: Permission denied)
    events/2     17       root  txt   unknown                                        /proc/17/exe (readlink: Permission denied)
    events/2     17       root NOFD                                                  /proc/17/fd (opendir: Permission denied)
    events/3     18       root  cwd   unknown                                        /proc/18/cwd (readlink: Permission denied)
    events/3     18       root  rtd   unknown                                        /proc/18/root (readlink: Permission denied)
    events/3     18       root  txt   unknown                                        /proc/18/exe (readlink: Permission denied)
    events/3     18       root NOFD                                                  /proc/18/fd (opendir: Permission denied)
    khelper      19       root  cwd   unknown                                        /proc/19/cwd (readlink: Permission denied)
    khelper      19       root  rtd   unknown                                        /proc/19/root (readlink: Permission denied)
    khelper      19       root  txt   unknown                                        /proc/19/exe (readlink: Permission denied)
    khelper      19       root NOFD                                                  /proc/19/fd (opendir: Permission denied)
    kstop/0      20       root  cwd   unknown                                        /proc/20/cwd (readlink: Permission denied)
    kstop/0      20       root  rtd   unknown                                        /proc/20/root (readlink: Permission denied)
    kstop/0      20       root  txt   unknown                                        /proc/20/exe (readlink: Permission denied)
    kstop/0      20       root NOFD                                                  /proc/20/fd (opendir: Permission denied)
    kstop/1      21       root  cwd   unknown                                        /proc/21/cwd (readlink: Permission denied)
    kstop/1      21       root  rtd   unknown                                        /proc/21/root (readlink: Permission denied)
    kstop/1      21       root  txt   unknown                                        /proc/21/exe (readlink: Permission denied)
    kstop/1      21       root NOFD                                                  /proc/21/fd (opendir: Permission denied)
    kstop/2      22       root  cwd   unknown                                        /proc/22/cwd (readlink: Permission denied)
    kstop/2      22       root  rtd   unknown                                        /proc/22/root (readlink: Permission denied)
    kstop/2      22       root  txt   unknown                                        /proc/22/exe (readlink: Permission denied)
    kstop/2      22       root NOFD                                                  /proc/22/fd (opendir: Permission denied)
    kstop/3      23       root  cwd   unknown                                        /proc/23/cwd (readlink: Permission denied)
    kstop/3      23       root  rtd   unknown                                        /proc/23/root (readlink: Permission denied)
    kstop/3      23       root  txt   unknown                                        /proc/23/exe (readlink: Permission denied)
    kstop/3      23       root NOFD                                                  /proc/23/fd (opendir: Permission denied)
    kintegrit    24       root  cwd   unknown                                        /proc/24/cwd (readlink: Permission denied)
    kintegrit    24       root  rtd   unknown                                        /proc/24/root (readlink: Permission denied)
    kintegrit    24       root  txt   unknown                                        /proc/24/exe (readlink: Permission denied)
    kintegrit    24       root NOFD                                                  /proc/24/fd (opendir: Permission denied)
    kintegrit    25       root  cwd   unknown                                        /proc/25/cwd (readlink: Permission denied)
    kintegrit    25       root  rtd   unknown                                        /proc/25/root (readlink: Permission denied)
    kintegrit    25       root  txt   unknown                                        /proc/25/exe (readlink: Permission denied)
    kintegrit    25       root NOFD                                                  /proc/25/fd (opendir: Permission denied)
    kintegrit    26       root  cwd   unknown                                        /proc/26/cwd (readlink: Permission denied)
    kintegrit    26       root  rtd   unknown                                        /proc/26/root (readlink: Permission denied)
    kintegrit    26       root  txt   unknown                                        /proc/26/exe (readlink: Permission denied)
    kintegrit    26       root NOFD                                                  /proc/26/fd (opendir: Permission denied)
    kintegrit    27       root  cwd   unknown                                        /proc/27/cwd (readlink: Permission denied)
    kintegrit    27       root  rtd   unknown                                        /proc/27/root (readlink: Permission denied)
    kintegrit    27       root  txt   unknown                                        /proc/27/exe (readlink: Permission denied)
    kintegrit    27       root NOFD                                                  /proc/27/fd (opendir: Permission denied)
    kblockd/0    28       root  cwd   unknown                                        /proc/28/cwd (readlink: Permission denied)
    kblockd/0    28       root  rtd   unknown                                        /proc/28/root (readlink: Permission denied)
    kblockd/0    28       root  txt   unknown                                        /proc/28/exe (readlink: Permission denied)
    kblockd/0    28       root NOFD                                                  /proc/28/fd (opendir: Permission denied)
    kblockd/1    29       root  cwd   unknown                                        /proc/29/cwd (readlink: Permission denied)
    kblockd/1    29       root  rtd   unknown                                        /proc/29/root (readlink: Permission denied)
    kblockd/1    29       root  txt   unknown                                        /proc/29/exe (readlink: Permission denied)
    kblockd/1    29       root NOFD                                                  /proc/29/fd (opendir: Permission denied)
    kblockd/2    30       root  cwd   unknown                                        /proc/30/cwd (readlink: Permission denied)
    kblockd/2    30       root  rtd   unknown                                        /proc/30/root (readlink: Permission denied)
    kblockd/2    30       root  txt   unknown                                        /proc/30/exe (readlink: Permission denied)
    kblockd/2    30       root NOFD                                                  /proc/30/fd (opendir: Permission denied)
    kblockd/3    31       root  cwd   unknown                                        /proc/31/cwd (readlink: Permission denied)
    kblockd/3    31       root  rtd   unknown                                        /proc/31/root (readlink: Permission denied)
    kblockd/3    31       root  txt   unknown                                        /proc/31/exe (readlink: Permission denied)
    kblockd/3    31       root NOFD                                                  /proc/31/fd (opendir: Permission denied)
    kacpid       32       root  cwd   unknown                                        /proc/32/cwd (readlink: Permission denied)
    kacpid       32       root  rtd   unknown                                        /proc/32/root (readlink: Permission denied)
    kacpid       32       root  txt   unknown                                        /proc/32/exe (readlink: Permission denied)
    kacpid       32       root NOFD                                                  /proc/32/fd (opendir: Permission denied)
    kacpi_not    33       root  cwd   unknown                                        /proc/33/cwd (readlink: Permission denied)
    kacpi_not    33       root  rtd   unknown                                        /proc/33/root (readlink: Permission denied)
    kacpi_not    33       root  txt   unknown                                        /proc/33/exe (readlink: Permission denied)
    kacpi_not    33       root NOFD                                                  /proc/33/fd (opendir: Permission denied)
    cqueue       34       root  cwd   unknown                                        /proc/34/cwd (readlink: Permission denied)
    cqueue       34       root  rtd   unknown                                        /proc/34/root (readlink: Permission denied)
    cqueue       34       root  txt   unknown                                        /proc/34/exe (readlink: Permission denied)
    cqueue       34       root NOFD                                                  /proc/34/fd (opendir: Permission denied)
    ata/0        35       root  cwd   unknown                                        /proc/35/cwd (readlink: Permission denied)
    ata/0        35       root  rtd   unknown                                        /proc/35/root (readlink: Permission denied)
    ata/0        35       root  txt   unknown                                        /proc/35/exe (readlink: Permission denied)
    ata/0        35       root NOFD                                                  /proc/35/fd (opendir: Permission denied)
    ata/1        36       root  cwd   unknown                                        /proc/36/cwd (readlink: Permission denied)
    ata/1        36       root  rtd   unknown                                        /proc/36/root (readlink: Permission denied)
    ata/1        36       root  txt   unknown                                        /proc/36/exe (readlink: Permission denied)
    ata/1        36       root NOFD                                                  /proc/36/fd (opendir: Permission denied)
    ata/2        37       root  cwd   unknown                                        /proc/37/cwd (readlink: Permission denied)
    ata/2        37       root  rtd   unknown                                        /proc/37/root (readlink: Permission denied)
    ata/2        37       root  txt   unknown                                        /proc/37/exe (readlink: Permission denied)
    ata/2        37       root NOFD                                                  /proc/37/fd (opendir: Permission denied)
    ata/3        38       root  cwd   unknown                                        /proc/38/cwd (readlink: Permission denied)
    ata/3        38       root  rtd   unknown                                        /proc/38/root (readlink: Permission denied)
    ata/3        38       root  txt   unknown                                        /proc/38/exe (readlink: Permission denied)
    ata/3        38       root NOFD                                                  /proc/38/fd (opendir: Permission denied)
    ata_aux      39       root  cwd   unknown                                        /proc/39/cwd (readlink: Permission denied)
    ata_aux      39       root  rtd   unknown                                        /proc/39/root (readlink: Permission denied)
    ata_aux      39       root  txt   unknown                                        /proc/39/exe (readlink: Permission denied)
    ata_aux      39       root NOFD                                                  /proc/39/fd (opendir: Permission denied)
    ksuspend_    40       root  cwd   unknown                                        /proc/40/cwd (readlink: Permission denied)
    ksuspend_    40       root  rtd   unknown                                        /proc/40/root (readlink: Permission denied)
    ksuspend_    40       root  txt   unknown                                        /proc/40/exe (readlink: Permission denied)
    ksuspend_    40       root NOFD                                                  /proc/40/fd (opendir: Permission denied)
    khubd        41       root  cwd   unknown                                        /proc/41/cwd (readlink: Permission denied)
    khubd        41       root  rtd   unknown                                        /proc/41/root (readlink: Permission denied)
    khubd        41       root  txt   unknown                                        /proc/41/exe (readlink: Permission denied)
    khubd        41       root NOFD                                                  /proc/41/fd (opendir: Permission denied)
    kseriod      42       root  cwd   unknown                                        /proc/42/cwd (readlink: Permission denied)
    kseriod      42       root  rtd   unknown                                        /proc/42/root (readlink: Permission denied)
    kseriod      42       root  txt   unknown                                        /proc/42/exe (readlink: Permission denied)
    kseriod      42       root NOFD                                                  /proc/42/fd (opendir: Permission denied)
    kmmcd        43       root  cwd   unknown                                        /proc/43/cwd (readlink: Permission denied)
    kmmcd        43       root  rtd   unknown                                        /proc/43/root (readlink: Permission denied)
    kmmcd        43       root  txt   unknown                                        /proc/43/exe (readlink: Permission denied)
    kmmcd        43       root NOFD                                                  /proc/43/fd (opendir: Permission denied)
    btaddconn    44       root  cwd   unknown                                        /proc/44/cwd (readlink: Permission denied)
    btaddconn    44       root  rtd   unknown                                        /proc/44/root (readlink: Permission denied)
    btaddconn    44       root  txt   unknown                                        /proc/44/exe (readlink: Permission denied)
    btaddconn    44       root NOFD                                                  /proc/44/fd (opendir: Permission denied)
    btdelconn    45       root  cwd   unknown                                        /proc/45/cwd (readlink: Permission denied)
    btdelconn    45       root  rtd   unknown                                        /proc/45/root (readlink: Permission denied)
    btdelconn    45       root  txt   unknown                                        /proc/45/exe (readlink: Permission denied)
    btdelconn    45       root NOFD                                                  /proc/45/fd (opendir: Permission denied)
    pdflush      46       root  cwd   unknown                                        /proc/46/cwd (readlink: Permission denied)
    pdflush      46       root  rtd   unknown                                        /proc/46/root (readlink: Permission denied)
    pdflush      46       root  txt   unknown                                        /proc/46/exe (readlink: Permission denied)
    pdflush      46       root NOFD                                                  /proc/46/fd (opendir: Permission denied)
    pdflush      47       root  cwd   unknown                                        /proc/47/cwd (readlink: Permission denied)
    pdflush      47       root  rtd   unknown                                        /proc/47/root (readlink: Permission denied)
    pdflush      47       root  txt   unknown                                        /proc/47/exe (readlink: Permission denied)
    pdflush      47       root NOFD                                                  /proc/47/fd (opendir: Permission denied)
    kswapd0      48       root  cwd   unknown                                        /proc/48/cwd (readlink: Permission denied)
    kswapd0      48       root  rtd   unknown                                        /proc/48/root (readlink: Permission denied)
    kswapd0      48       root  txt   unknown                                        /proc/48/exe (readlink: Permission denied)
    kswapd0      48       root NOFD                                                  /proc/48/fd (opendir: Permission denied)
    aio/0        49       root  cwd   unknown                                        /proc/49/cwd (readlink: Permission denied)
    aio/0        49       root  rtd   unknown                                        /proc/49/root (readlink: Permission denied)
    aio/0        49       root  txt   unknown                                        /proc/49/exe (readlink: Permission denied)
    aio/0        49       root NOFD                                                  /proc/49/fd (opendir: Permission denied)
    aio/1        50       root  cwd   unknown                                        /proc/50/cwd (readlink: Permission denied)
    aio/1        50       root  rtd   unknown                                        /proc/50/root (readlink: Permission denied)
    aio/1        50       root  txt   unknown                                        /proc/50/exe (readlink: Permission denied)
    aio/1        50       root NOFD                                                  /proc/50/fd (opendir: Permission denied)
    aio/2        51       root  cwd   unknown                                        /proc/51/cwd (readlink: Permission denied)
    aio/2        51       root  rtd   unknown                                        /proc/51/root (readlink: Permission denied)
    aio/2        51       root  txt   unknown                                        /proc/51/exe (readlink: Permission denied)
    aio/2        51       root NOFD                                                  /proc/51/fd (opendir: Permission denied)
    aio/3        52       root  cwd   unknown                                        /proc/52/cwd (readlink: Permission denied)
    aio/3        52       root  rtd   unknown                                        /proc/52/root (readlink: Permission denied)
    aio/3        52       root  txt   unknown                                        /proc/52/exe (readlink: Permission denied)
    aio/3        52       root NOFD                                                  /proc/52/fd (opendir: Permission denied)
    ecryptfs-    53       root  cwd   unknown                                        /proc/53/cwd (readlink: Permission denied)
    ecryptfs-    53       root  rtd   unknown                                        /proc/53/root (readlink: Permission denied)
    ecryptfs-    53       root  txt   unknown                                        /proc/53/exe (readlink: Permission denied)
    ecryptfs-    53       root NOFD                                                  /proc/53/fd (opendir: Permission denied)
    scsi_eh_0    56       root  cwd   unknown                                        /proc/56/cwd (readlink: Permission denied)
    scsi_eh_0    56       root  rtd   unknown                                        /proc/56/root (readlink: Permission denied)
    scsi_eh_0    56       root  txt   unknown                                        /proc/56/exe (readlink: Permission denied)
    scsi_eh_0    56       root NOFD                                                  /proc/56/fd (opendir: Permission denied)
    scsi_eh_1    57       root  cwd   unknown                                        /proc/57/cwd (readlink: Permission denied)
    scsi_eh_1    57       root  rtd   unknown                                        /proc/57/root (readlink: Permission denied)
    scsi_eh_1    57       root  txt   unknown                                        /proc/57/exe (readlink: Permission denied)
    scsi_eh_1    57       root NOFD                                                  /proc/57/fd (opendir: Permission denied)
    scsi_eh_2    58       root  cwd   unknown                                        /proc/58/cwd (readlink: Permission denied)
    scsi_eh_2    58       root  rtd   unknown                                        /proc/58/root (readlink: Permission denied)
    scsi_eh_2    58       root  txt   unknown                                        /proc/58/exe (readlink: Permission denied)
    scsi_eh_2    58       root NOFD                                                  /proc/58/fd (opendir: Permission denied)
    scsi_eh_3    59       root  cwd   unknown                                        /proc/59/cwd (readlink: Permission denied)
    scsi_eh_3    59       root  rtd   unknown                                        /proc/59/root (readlink: Permission denied)
    scsi_eh_3    59       root  txt   unknown                                        /proc/59/exe (readlink: Permission denied)
    scsi_eh_3    59       root NOFD                                                  /proc/59/fd (opendir: Permission denied)
    scsi_eh_4    60       root  cwd   unknown                                        /proc/60/cwd (readlink: Permission denied)
    scsi_eh_4    60       root  rtd   unknown                                        /proc/60/root (readlink: Permission denied)
    scsi_eh_4    60       root  txt   unknown                                        /proc/60/exe (readlink: Permission denied)
    scsi_eh_4    60       root NOFD                                                  /proc/60/fd (opendir: Permission denied)
    scsi_eh_5    61       root  cwd   unknown                                        /proc/61/cwd (readlink: Permission denied)
    scsi_eh_5    61       root  rtd   unknown                                        /proc/61/root (readlink: Permission denied)
    scsi_eh_5    61       root  txt   unknown                                        /proc/61/exe (readlink: Permission denied)
    scsi_eh_5    61       root NOFD                                                  /proc/61/fd (opendir: Permission denied)
    kstriped     62       root  cwd   unknown                                        /proc/62/cwd (readlink: Permission denied)
    kstriped     62       root  rtd   unknown                                        /proc/62/root (readlink: Permission denied)
    kstriped     62       root  txt   unknown                                        /proc/62/exe (readlink: Permission denied)
    kstriped     62       root NOFD                                                  /proc/62/fd (opendir: Permission denied)
    kmpathd/0    63       root  cwd   unknown                                        /proc/63/cwd (readlink: Permission denied)
    kmpathd/0    63       root  rtd   unknown                                        /proc/63/root (readlink: Permission denied)
    kmpathd/0    63       root  txt   unknown                                        /proc/63/exe (readlink: Permission denied)
    kmpathd/0    63       root NOFD                                                  /proc/63/fd (opendir: Permission denied)
    kmpathd/1    64       root  cwd   unknown                                        /proc/64/cwd (readlink: Permission denied)
    kmpathd/1    64       root  rtd   unknown                                        /proc/64/root (readlink: Permission denied)
    kmpathd/1    64       root  txt   unknown                                        /proc/64/exe (readlink: Permission denied)
    kmpathd/1    64       root NOFD                                                  /proc/64/fd (opendir: Permission denied)
    kmpathd/2    65       root  cwd   unknown                                        /proc/65/cwd (readlink: Permission denied)
    kmpathd/2    65       root  rtd   unknown                                        /proc/65/root (readlink: Permission denied)
    kmpathd/2    65       root  txt   unknown                                        /proc/65/exe (readlink: Permission denied)
    kmpathd/2    65       root NOFD                                                  /proc/65/fd (opendir: Permission denied)
    kmpathd/3    66       root  cwd   unknown                                        /proc/66/cwd (readlink: Permission denied)
    kmpathd/3    66       root  rtd   unknown                                        /proc/66/root (readlink: Permission denied)
    kmpathd/3    66       root  txt   unknown                                        /proc/66/exe (readlink: Permission denied)
    kmpathd/3    66       root NOFD                                                  /proc/66/fd (opendir: Permission denied)
    kmpath_ha    67       root  cwd   unknown                                        /proc/67/cwd (readlink: Permission denied)
    kmpath_ha    67       root  rtd   unknown                                        /proc/67/root (readlink: Permission denied)
    kmpath_ha    67       root  txt   unknown                                        /proc/67/exe (readlink: Permission denied)
    kmpath_ha    67       root NOFD                                                  /proc/67/fd (opendir: Permission denied)
    ksnapd       68       root  cwd   unknown                                        /proc/68/cwd (readlink: Permission denied)
    ksnapd       68       root  rtd   unknown                                        /proc/68/root (readlink: Permission denied)
    ksnapd       68       root  txt   unknown                                        /proc/68/exe (readlink: Permission denied)
    ksnapd       68       root NOFD                                                  /proc/68/fd (opendir: Permission denied)
    kondemand    69       root  cwd   unknown                                        /proc/69/cwd (readlink: Permission denied)
    kondemand    69       root  rtd   unknown                                        /proc/69/root (readlink: Permission denied)
    kondemand    69       root  txt   unknown                                        /proc/69/exe (readlink: Permission denied)
    kondemand    69       root NOFD                                                  /proc/69/fd (opendir: Permission denied)
    kondemand    70       root  cwd   unknown                                        /proc/70/cwd (readlink: Permission denied)
    kondemand    70       root  rtd   unknown                                        /proc/70/root (readlink: Permission denied)
    kondemand    70       root  txt   unknown                                        /proc/70/exe (readlink: Permission denied)
    kondemand    70       root NOFD                                                  /proc/70/fd (opendir: Permission denied)
    kondemand    71       root  cwd   unknown                                        /proc/71/cwd (readlink: Permission denied)
    kondemand    71       root  rtd   unknown                                        /proc/71/root (readlink: Permission denied)
    kondemand    71       root  txt   unknown                                        /proc/71/exe (readlink: Permission denied)
    kondemand    71       root NOFD                                                  /proc/71/fd (opendir: Permission denied)
    kondemand    72       root  cwd   unknown                                        /proc/72/cwd (readlink: Permission denied)
    kondemand    72       root  rtd   unknown                                        /proc/72/root (readlink: Permission denied)
    kondemand    72       root  txt   unknown                                        /proc/72/exe (readlink: Permission denied)
    kondemand    72       root NOFD                                                  /proc/72/fd (opendir: Permission denied)
    krfcommd     73       root  cwd   unknown                                        /proc/73/cwd (readlink: Permission denied)
    krfcommd     73       root  rtd   unknown                                        /proc/73/root (readlink: Permission denied)
    krfcommd     73       root  txt   unknown                                        /proc/73/exe (readlink: Permission denied)
    krfcommd     73       root NOFD                                                  /proc/73/fd (opendir: Permission denied)
    mpt_poll_   359       root  cwd   unknown                                        /proc/359/cwd (readlink: Permission denied)
    mpt_poll_   359       root  rtd   unknown                                        /proc/359/root (readlink: Permission denied)
    mpt_poll_   359       root  txt   unknown                                        /proc/359/exe (readlink: Permission denied)
    mpt_poll_   359       root NOFD                                                  /proc/359/fd (opendir: Permission denied)
    scsi_eh_6   910       root  cwd   unknown                                        /proc/910/cwd (readlink: Permission denied)
    scsi_eh_6   910       root  rtd   unknown                                        /proc/910/root (readlink: Permission denied)
    scsi_eh_6   910       root  txt   unknown                                        /proc/910/exe (readlink: Permission denied)
    scsi_eh_6   910       root NOFD                                                  /proc/910/fd (opendir: Permission denied)
    kdmflush    966       root  cwd   unknown                                        /proc/966/cwd (readlink: Permission denied)
    kdmflush    966       root  rtd   unknown                                        /proc/966/root (readlink: Permission denied)
    kdmflush    966       root  txt   unknown                                        /proc/966/exe (readlink: Permission denied)
    kdmflush    966       root NOFD                                                  /proc/966/fd (opendir: Permission denied)
    kdmflush    971       root  cwd   unknown                                        /proc/971/cwd (readlink: Permission denied)
    kdmflush    971       root  rtd   unknown                                        /proc/971/root (readlink: Permission denied)
    kdmflush    971       root  txt   unknown                                        /proc/971/exe (readlink: Permission denied)
    kdmflush    971       root NOFD                                                  /proc/971/fd (opendir: Permission denied)
    kjournald  1009       root  cwd   unknown                                        /proc/1009/cwd (readlink: Permission denied)
    kjournald  1009       root  rtd   unknown                                        /proc/1009/root (readlink: Permission denied)
    kjournald  1009       root  txt   unknown                                        /proc/1009/exe (readlink: Permission denied)
    kjournald  1009       root NOFD                                                  /proc/1009/fd (opendir: Permission denied)
    udevd      1129       root  cwd   unknown                                        /proc/1129/cwd (readlink: Permission denied)
    udevd      1129       root  rtd   unknown                                        /proc/1129/root (readlink: Permission denied)
    udevd      1129       root  txt   unknown                                        /proc/1129/exe (readlink: Permission denied)
    udevd      1129       root NOFD                                                  /proc/1129/fd (opendir: Permission denied)
    kjournald  2013       root  cwd   unknown                                        /proc/2013/cwd (readlink: Permission denied)
    kjournald  2013       root  rtd   unknown                                        /proc/2013/root (readlink: Permission denied)
    kjournald  2013       root  txt   unknown                                        /proc/2013/exe (readlink: Permission denied)
    kjournald  2013       root NOFD                                                  /proc/2013/fd (opendir: Permission denied)
    portmap    2149     daemon  cwd   unknown                                        /proc/2149/cwd (readlink: Permission denied)
    portmap    2149     daemon  rtd   unknown                                        /proc/2149/root (readlink: Permission denied)
    portmap    2149     daemon  txt   unknown                                        /proc/2149/exe (readlink: Permission denied)
    portmap    2149     daemon NOFD                                                  /proc/2149/fd (opendir: Permission denied)
    rpc.statd  2165      statd  cwd   unknown                                        /proc/2165/cwd (readlink: Permission denied)
    rpc.statd  2165      statd  rtd   unknown                                        /proc/2165/root (readlink: Permission denied)
    rpc.statd  2165      statd  txt   unknown                                        /proc/2165/exe (readlink: Permission denied)
    rpc.statd  2165      statd NOFD                                                  /proc/2165/fd (opendir: Permission denied)
    getty      2326       root  cwd   unknown                                        /proc/2326/cwd (readlink: Permission denied)
    getty      2326       root  rtd   unknown                                        /proc/2326/root (readlink: Permission denied)
    getty      2326       root  txt   unknown                                        /proc/2326/exe (readlink: Permission denied)
    getty      2326       root NOFD                                                  /proc/2326/fd (opendir: Permission denied)
    getty      2327       root  cwd   unknown                                        /proc/2327/cwd (readlink: Permission denied)
    getty      2327       root  rtd   unknown                                        /proc/2327/root (readlink: Permission denied)
    getty      2327       root  txt   unknown                                        /proc/2327/exe (readlink: Permission denied)
    getty      2327       root NOFD                                                  /proc/2327/fd (opendir: Permission denied)
    getty      2330       root  cwd   unknown                                        /proc/2330/cwd (readlink: Permission denied)
    getty      2330       root  rtd   unknown                                        /proc/2330/root (readlink: Permission denied)
    getty      2330       root  txt   unknown                                        /proc/2330/exe (readlink: Permission denied)
    getty      2330       root NOFD                                                  /proc/2330/fd (opendir: Permission denied)
    getty      2331       root  cwd   unknown                                        /proc/2331/cwd (readlink: Permission denied)
    getty      2331       root  rtd   unknown                                        /proc/2331/root (readlink: Permission denied)
    getty      2331       root  txt   unknown                                        /proc/2331/exe (readlink: Permission denied)
    getty      2331       root NOFD                                                  /proc/2331/fd (opendir: Permission denied)
    getty      2333       root  cwd   unknown                                        /proc/2333/cwd (readlink: Permission denied)
    getty      2333       root  rtd   unknown                                        /proc/2333/root (readlink: Permission denied)
    getty      2333       root  txt   unknown                                        /proc/2333/exe (readlink: Permission denied)
    getty      2333       root NOFD                                                  /proc/2333/fd (opendir: Permission denied)
    acpid      2416       root  cwd   unknown                                        /proc/2416/cwd (readlink: Permission denied)
    acpid      2416       root  rtd   unknown                                        /proc/2416/root (readlink: Permission denied)
    acpid      2416       root  txt   unknown                                        /proc/2416/exe (readlink: Permission denied)
    acpid      2416       root NOFD                                                  /proc/2416/fd (opendir: Permission denied)
    syslogd    2453     syslog  cwd   unknown                                        /proc/2453/cwd (readlink: Permission denied)
    syslogd    2453     syslog  rtd   unknown                                        /proc/2453/root (readlink: Permission denied)
    syslogd    2453     syslog  txt   unknown                                        /proc/2453/exe (readlink: Permission denied)
    syslogd    2453     syslog NOFD                                                  /proc/2453/fd (opendir: Permission denied)
    dd         2471       root  cwd   unknown                                        /proc/2471/cwd (readlink: Permission denied)
    dd         2471       root  rtd   unknown                                        /proc/2471/root (readlink: Permission denied)
    dd         2471       root  txt   unknown                                        /proc/2471/exe (readlink: Permission denied)
    dd         2471       root NOFD                                                  /proc/2471/fd (opendir: Permission denied)
    klogd      2473       klog  cwd   unknown                                        /proc/2473/cwd (readlink: Permission denied)
    klogd      2473       klog  rtd   unknown                                        /proc/2473/root (readlink: Permission denied)
    klogd      2473       klog  txt   unknown                                        /proc/2473/exe (readlink: Permission denied)
    klogd      2473       klog NOFD                                                  /proc/2473/fd (opendir: Permission denied)
    dbus-daem  2491 messagebus  cwd   unknown                                        /proc/2491/cwd (readlink: Permission denied)
    dbus-daem  2491 messagebus  rtd   unknown                                        /proc/2491/root (readlink: Permission denied)
    dbus-daem  2491 messagebus  txt   unknown                                        /proc/2491/exe (readlink: Permission denied)
    dbus-daem  2491 messagebus NOFD                                                  /proc/2491/fd (opendir: Permission denied)
    console-k  2529       root  cwd   unknown                                        /proc/2529/cwd (readlink: Permission denied)
    console-k  2529       root  rtd   unknown                                        /proc/2529/root (readlink: Permission denied)
    console-k  2529       root  txt   unknown                                        /proc/2529/exe (readlink: Permission denied)
    console-k  2529       root NOFD                                                  /proc/2529/fd (opendir: Permission denied)
    rpciod/0   2695       root  cwd   unknown                                        /proc/2695/cwd (readlink: Permission denied)
    rpciod/0   2695       root  rtd   unknown                                        /proc/2695/root (readlink: Permission denied)
    rpciod/0   2695       root  txt   unknown                                        /proc/2695/exe (readlink: Permission denied)
    rpciod/0   2695       root NOFD                                                  /proc/2695/fd (opendir: Permission denied)
    rpciod/1   2698       root  cwd   unknown                                        /proc/2698/cwd (readlink: Permission denied)
    rpciod/1   2698       root  rtd   unknown                                        /proc/2698/root (readlink: Permission denied)
    rpciod/1   2698       root  txt   unknown                                        /proc/2698/exe (readlink: Permission denied)
    rpciod/1   2698       root NOFD                                                  /proc/2698/fd (opendir: Permission denied)
    rpciod/2   2699       root  cwd   unknown                                        /proc/2699/cwd (readlink: Permission denied)
    rpciod/2   2699       root  rtd   unknown                                        /proc/2699/root (readlink: Permission denied)
    rpciod/2   2699       root  txt   unknown                                        /proc/2699/exe (readlink: Permission denied)
    rpciod/2   2699       root NOFD                                                  /proc/2699/fd (opendir: Permission denied)
    rpciod/3   2700       root  cwd   unknown                                        /proc/2700/cwd (readlink: Permission denied)
    rpciod/3   2700       root  rtd   unknown                                        /proc/2700/root (readlink: Permission denied)
    rpciod/3   2700       root  txt   unknown                                        /proc/2700/exe (readlink: Permission denied)
    rpciod/3   2700       root NOFD                                                  /proc/2700/fd (opendir: Permission denied)
    nfsiod     2703       root  cwd   unknown                                        /proc/2703/cwd (readlink: Permission denied)
    nfsiod     2703       root  rtd   unknown                                        /proc/2703/root (readlink: Permission denied)
    nfsiod     2703       root  txt   unknown                                        /proc/2703/exe (readlink: Permission denied)
    nfsiod     2703       root NOFD                                                  /proc/2703/fd (opendir: Permission denied)
    lockd      2708       root  cwd   unknown                                        /proc/2708/cwd (readlink: Permission denied)
    lockd      2708       root  rtd   unknown                                        /proc/2708/root (readlink: Permission denied)
    lockd      2708       root  txt   unknown                                        /proc/2708/exe (readlink: Permission denied)
    lockd      2708       root NOFD                                                  /proc/2708/fd (opendir: Permission denied)
    sshd       2734       root  cwd   unknown                                        /proc/2734/cwd (readlink: Permission denied)
    sshd       2734       root  rtd   unknown                                        /proc/2734/root (readlink: Permission denied)
    sshd       2734       root  txt   unknown                                        /proc/2734/exe (readlink: Permission denied)
    sshd       2734       root NOFD                                                  /proc/2734/fd (opendir: Permission denied)
    zabbix_ag  2776     zabbix  cwd   unknown                                        /proc/2776/cwd (readlink: Permission denied)
    zabbix_ag  2776     zabbix  rtd   unknown                                        /proc/2776/root (readlink: Permission denied)
    zabbix_ag  2776     zabbix  txt   unknown                                        /proc/2776/exe (readlink: Permission denied)
    zabbix_ag  2776     zabbix NOFD                                                  /proc/2776/fd (opendir: Permission denied)
    zabbix_ag  2778     zabbix  cwd   unknown                                        /proc/2778/cwd (readlink: Permission denied)
    zabbix_ag  2778     zabbix  rtd   unknown                                        /proc/2778/root (readlink: Permission denied)
    zabbix_ag  2778     zabbix  txt   unknown                                        /proc/2778/exe (readlink: Permission denied)
    zabbix_ag  2778     zabbix NOFD                                                  /proc/2778/fd (opendir: Permission denied)
    zabbix_ag  2779     zabbix  cwd   unknown                                        /proc/2779/cwd (readlink: Permission denied)
    zabbix_ag  2779     zabbix  rtd   unknown                                        /proc/2779/root (readlink: Permission denied)
    zabbix_ag  2779     zabbix  txt   unknown                                        /proc/2779/exe (readlink: Permission denied)
    zabbix_ag  2779     zabbix NOFD                                                  /proc/2779/fd (opendir: Permission denied)
    zabbix_ag  2780     zabbix  cwd   unknown                                        /proc/2780/cwd (readlink: Permission denied)
    zabbix_ag  2780     zabbix  rtd   unknown                                        /proc/2780/root (readlink: Permission denied)
    zabbix_ag  2780     zabbix  txt   unknown                                        /proc/2780/exe (readlink: Permission denied)
    zabbix_ag  2780     zabbix NOFD                                                  /proc/2780/fd (opendir: Permission denied)
    zabbix_ag  2781     zabbix  cwd   unknown                                        /proc/2781/cwd (readlink: Permission denied)
    zabbix_ag  2781     zabbix  rtd   unknown                                        /proc/2781/root (readlink: Permission denied)
    zabbix_ag  2781     zabbix  txt   unknown                                        /proc/2781/exe (readlink: Permission denied)
    zabbix_ag  2781     zabbix NOFD                                                  /proc/2781/fd (opendir: Permission denied)
    zabbix_ag  2782     zabbix  cwd   unknown                                        /proc/2782/cwd (readlink: Permission denied)
    zabbix_ag  2782     zabbix  rtd   unknown                                        /proc/2782/root (readlink: Permission denied)
    zabbix_ag  2782     zabbix  txt   unknown                                        /proc/2782/exe (readlink: Permission denied)
    zabbix_ag  2782     zabbix NOFD                                                  /proc/2782/fd (opendir: Permission denied)
    avahi-dae  2795      avahi  cwd   unknown                                        /proc/2795/cwd (readlink: Permission denied)
    avahi-dae  2795      avahi  rtd   unknown                                        /proc/2795/root (readlink: Permission denied)
    avahi-dae  2795      avahi  txt   unknown                                        /proc/2795/exe (readlink: Permission denied)
    avahi-dae  2795      avahi NOFD                                                  /proc/2795/fd (opendir: Permission denied)
    avahi-dae  2796      avahi  cwd   unknown                                        /proc/2796/cwd (readlink: Permission denied)
    avahi-dae  2796      avahi  rtd   unknown                                        /proc/2796/root (readlink: Permission denied)
    avahi-dae  2796      avahi  txt   unknown                                        /proc/2796/exe (readlink: Permission denied)
    avahi-dae  2796      avahi NOFD                                                  /proc/2796/fd (opendir: Permission denied)
    atd        2846     daemon  cwd   unknown                                        /proc/2846/cwd (readlink: Permission denied)
    atd        2846     daemon  rtd   unknown                                        /proc/2846/root (readlink: Permission denied)
    atd        2846     daemon  txt   unknown                                        /proc/2846/exe (readlink: Permission denied)
    atd        2846     daemon NOFD                                                  /proc/2846/fd (opendir: Permission denied)
    cron       2871       root  cwd   unknown                                        /proc/2871/cwd (readlink: Permission denied)
    cron       2871       root  rtd   unknown                                        /proc/2871/root (readlink: Permission denied)
    cron       2871       root  txt   unknown                                        /proc/2871/exe (readlink: Permission denied)
    cron       2871       root NOFD                                                  /proc/2871/fd (opendir: Permission denied)
    getty      2890       root  cwd   unknown                                        /proc/2890/cwd (readlink: Permission denied)
    getty      2890       root  rtd   unknown                                        /proc/2890/root (readlink: Permission denied)
    getty      2890       root  txt   unknown                                        /proc/2890/exe (readlink: Permission denied)
    getty      2890       root NOFD                                                  /proc/2890/fd (opendir: Permission denied)
    getty      2891       root  cwd   unknown                                        /proc/2891/cwd (readlink: Permission denied)
    getty      2891       root  rtd   unknown                                        /proc/2891/root (readlink: Permission denied)
    getty      2891       root  txt   unknown                                        /proc/2891/exe (readlink: Permission denied)
    getty      2891       root NOFD                                                  /proc/2891/fd (opendir: Permission denied)
    sshd      21294       root  cwd   unknown                                        /proc/21294/cwd (readlink: Permission denied)
    sshd      21294       root  rtd   unknown                                        /proc/21294/root (readlink: Permission denied)
    sshd      21294       root  txt   unknown                                        /proc/21294/exe (readlink: Permission denied)
    sshd      21294       root NOFD                                                  /proc/21294/fd (opendir: Permission denied)
    sshd      21302     evsadm  cwd   unknown                                        /proc/21302/cwd (readlink: Permission denied)
    sshd      21302     evsadm  rtd   unknown                                        /proc/21302/root (readlink: Permission denied)
    sshd      21302     evsadm  txt   unknown                                        /proc/21302/exe (readlink: Permission denied)
    sshd      21302     evsadm NOFD                                                  /proc/21302/fd (opendir: Permission denied)
    bash      21303     evsadm  cwd       DIR              252,1     4096      32774 /home/evsadm
    bash      21303     evsadm  rtd       DIR              252,1     4096          2 /
    bash      21303     evsadm  txt       REG              252,1   833592    1146948 /bin/bash
    bash      21303     evsadm  mem       REG              252,1    10192    1183601 /usr/lib/gconv/ISO8859-1.so
    bash      21303     evsadm  mem       REG              252,1    51616    1294591 /lib/libnss_files-2.9.so
    bash      21303     evsadm  mem       REG              252,1    43472    1294593 /lib/libnss_nis-2.9.so
    bash      21303     evsadm  mem       REG              252,1    97168    1294588 /lib/libnsl-2.9.so
    bash      21303     evsadm  mem       REG              252,1    35632    1294589 /lib/libnss_compat-2.9.so
    bash      21303     evsadm  mem       REG              252,1  1502512    1294582 /lib/libc-2.9.so
    bash      21303     evsadm  mem       REG              252,1    14608    1294585 /lib/libdl-2.9.so
    bash      21303     evsadm  mem       REG              252,1   249696    1294401 /lib/libncurses.so.5.7
    bash      21303     evsadm  mem       REG              252,1   135680    1294579 /lib/ld-2.9.so
    bash      21303     evsadm  mem       REG              252,1    26048    1180648 /usr/lib/gconv/gconv-modules.cache
    bash      21303     evsadm  mem       REG              252,1   221604    1205793 /usr/lib/locale/en_US/LC_CTYPE
    bash      21303     evsadm  mem       REG              252,1       59    1205794 /usr/lib/locale/en_US/LC_NUMERIC
    bash      21303     evsadm  mem       REG              252,1     2459    1205795 /usr/lib/locale/en_US/LC_TIME
    bash      21303     evsadm  mem       REG              252,1    19459    1205796 /usr/lib/locale/en_US/LC_COLLATE
    bash      21303     evsadm  mem       REG              252,1      291    1205797 /usr/lib/locale/en_US/LC_MONETARY
    bash      21303     evsadm  mem       REG              252,1       57    1205799 /usr/lib/locale/en_US/LC_MESSAGES/SYS_LC_MESSAGES
    bash      21303     evsadm  mem       REG              252,1       39    1205800 /usr/lib/locale/en_US/LC_PAPER
    bash      21303     evsadm  mem       REG              252,1       82    1205801 /usr/lib/locale/en_US/LC_NAME
    bash      21303     evsadm  mem       REG              252,1      160    1205802 /usr/lib/locale/en_US/LC_ADDRESS
    bash      21303     evsadm  mem       REG              252,1       64    1205803 /usr/lib/locale/en_US/LC_TELEPHONE
    bash      21303     evsadm  mem       REG              252,1       28    1205804 /usr/lib/locale/en_US/LC_MEASUREMENT
    bash      21303     evsadm  mem       REG              252,1      378    1205805 /usr/lib/locale/en_US/LC_IDENTIFICATION
    bash      21303     evsadm    0u      CHR              136,1                   3 /dev/pts/1
    bash      21303     evsadm    1u      CHR              136,1                   3 /dev/pts/1
    bash      21303     evsadm    2u      CHR              136,1                   3 /dev/pts/1
    bash      21303     evsadm  255u      CHR              136,1                   3 /dev/pts/1
    lsof      21466     evsadm  cwd       DIR              252,1     4096      32774 /home/evsadm
    lsof      21466     evsadm  rtd       DIR              252,1     4096          2 /
    lsof      21466     evsadm  txt       REG              252,1   118560    1184059 /usr/bin/lsof
    lsof      21466     evsadm  mem       REG              252,1  1502512    1294582 /lib/libc-2.9.so
    lsof      21466     evsadm  mem       REG              252,1   135680    1294579 /lib/ld-2.9.so
    lsof      21466     evsadm  mem       REG              252,1   221604    1205793 /usr/lib/locale/en_US/LC_CTYPE
    lsof      21466     evsadm    0u      CHR              136,1                   3 /dev/pts/1
    lsof      21466     evsadm    1w      REG              252,1       25      32793 /home/evsadm/staf_debug
    lsof      21466     evsadm    2u      CHR              136,1                   3 /dev/pts/1
    lsof      21466     evsadm    3r      DIR                0,3        0          1 /proc
    lsof      21466     evsadm    4r      DIR                0,3        0 4202140151 /proc/21466/fd
    lsof      21466     evsadm    5w     FIFO                0,6          4202140155 pipe
    lsof      21466     evsadm    6r     FIFO                0,6          4202140156 pipe
    lsof      21467     evsadm  cwd       DIR              252,1     4096      32774 /home/evsadm
    lsof      21467     evsadm  rtd       DIR              252,1     4096          2 /
    lsof      21467     evsadm  txt       REG              252,1   118560    1184059 /usr/bin/lsof
    lsof      21467     evsadm  mem       REG              252,1  1502512    1294582 /lib/libc-2.9.so
    lsof      21467     evsadm  mem       REG              252,1   135680    1294579 /lib/ld-2.9.so
    lsof      21467     evsadm  mem       REG              252,1   221604    1205793 /usr/lib/locale/en_US/LC_CTYPE
    lsof      21467     evsadm    4r     FIFO                0,6          4202140155 pipe
    lsof      21467     evsadm    7w     FIFO                0,6          4202140156 pipe
    bash      23119     evsadm  cwd       DIR              252,1     4096      32774 /home/evsadm
    bash      23119     evsadm  rtd       DIR              252,1     4096          2 /
    bash      23119     evsadm  txt       REG              252,1   833592    1146948 /bin/bash
    bash      23119     evsadm  mem       REG              252,1    10192    1183601 /usr/lib/gconv/ISO8859-1.so
    bash      23119     evsadm  mem       REG              252,1    51616    1294591 /lib/libnss_files-2.9.so
    bash      23119     evsadm  mem       REG              252,1    43472    1294593 /lib/libnss_nis-2.9.so
    bash      23119     evsadm  mem       REG              252,1    97168    1294588 /lib/libnsl-2.9.so
    bash      23119     evsadm  mem       REG              252,1    35632    1294589 /lib/libnss_compat-2.9.so
    bash      23119     evsadm  mem       REG              252,1  1502512    1294582 /lib/libc-2.9.so
    bash      23119     evsadm  mem       REG              252,1    14608    1294585 /lib/libdl-2.9.so
    bash      23119     evsadm  mem       REG              252,1   249696    1294401 /lib/libncurses.so.5.7
    bash      23119     evsadm  mem       REG              252,1   135680    1294579 /lib/ld-2.9.so
    bash      23119     evsadm  mem       REG              252,1    26048    1180648 /usr/lib/gconv/gconv-modules.cache
    bash      23119     evsadm  mem       REG              252,1   221604    1205793 /usr/lib/locale/en_US/LC_CTYPE
    bash      23119     evsadm  mem       REG              252,1       59    1205794 /usr/lib/locale/en_US/LC_NUMERIC
    bash      23119     evsadm  mem       REG              252,1     2459    1205795 /usr/lib/locale/en_US/LC_TIME
    bash      23119     evsadm  mem       REG              252,1    19459    1205796 /usr/lib/locale/en_US/LC_COLLATE
    bash      23119     evsadm  mem       REG              252,1      291    1205797 /usr/lib/locale/en_US/LC_MONETARY
    bash      23119     evsadm  mem       REG              252,1       57    1205799 /usr/lib/locale/en_US/LC_MESSAGES/SYS_LC_MESSAGES
    bash      23119     evsadm  mem       REG              252,1       39    1205800 /usr/lib/locale/en_US/LC_PAPER
    bash      23119     evsadm  mem       REG              252,1       82    1205801 /usr/lib/locale/en_US/LC_NAME
    bash      23119     evsadm  mem       REG              252,1      160    1205802 /usr/lib/locale/en_US/LC_ADDRESS
    bash      23119     evsadm  mem       REG              252,1       64    1205803 /usr/lib/locale/en_US/LC_TELEPHONE
    bash      23119     evsadm  mem       REG              252,1       28    1205804 /usr/lib/locale/en_US/LC_MEASUREMENT
    bash      23119     evsadm  mem       REG              252,1      378    1205805 /usr/lib/locale/en_US/LC_IDENTIFICATION
    bash      23119     evsadm    0r      CHR                1,3                3222 /dev/null
    bash      23119     evsadm    1u      CHR              136,0                   2 /dev/pts/0 (deleted)
    bash      23119     evsadm    2u      CHR              136,0                   2 /dev/pts/0 (deleted)
    STAFProc  23126     evsadm  cwd       DIR              252,1     4096      32774 /home/evsadm
    STAFProc  23126     evsadm  rtd       DIR              252,1     4096          2 /
    STAFProc  23126     evsadm  txt       REG              252,1  2917213    1254656 /usr/local/staf/bin/STAFProc
    STAFProc  23126     evsadm  mem       REG              252,1    88824    1294597 /lib/libresolv-2.9.so
    STAFProc  23126     evsadm  mem       REG              252,1    22848    1294590 /lib/libnss_dns-2.9.so
    STAFProc  23126     evsadm  mem       REG              252,1     9760    1294366 /lib/libnss_mdns4_minimal.so.2
    STAFProc  23126     evsadm  mem       REG              252,1   574242    1254842 /usr/local/staf/lib/libSTAFLog.so
    STAFProc  23126     evsadm  mem       REG              252,1   128713    1254850 /usr/local/staf/lib/libJSTAF.so
    STAFProc  23126     evsadm  mem       REG              252,1    33620    1254844 /usr/local/staf/lib/libSTAFDSLS.so
    STAFProc  23126     evsadm  mem       REG              252,1    51616    1294591 /lib/libnss_files-2.9.so
    STAFProc  23126     evsadm  mem       REG              252,1   157985    1254853 /usr/local/staf/lib/libSTAFTCP.so
    STAFProc  23126     evsadm  mem       REG              252,1    53726    1254847 /usr/local/staf/lib/libSTAFLIPC.so
    STAFProc  23126     evsadm  mem       REG              252,1    96768    1294474 /lib/libz.so.1.2.3.3
    STAFProc  23126     evsadm  mem       REG              252,1  1502512    1294582 /lib/libc-2.9.so
    STAFProc  23126     evsadm  mem       REG              252,1    96560    1294400 /lib/libgcc_s.so.1
    STAFProc  23126     evsadm  mem       REG              252,1   542928    1294586 /lib/libm-2.9.so
    STAFProc  23126     evsadm  mem       REG              252,1  1023448    1181022 /usr/lib/libstdc++.so.6.0.10
    STAFProc  23126     evsadm  mem       REG              252,1  1594824    1294410 /lib/libcrypto.so.0.9.8
    STAFProc  23126     evsadm  mem       REG              252,1   325120    1294415 /lib/libssl.so.0.9.8
    STAFProc  23126     evsadm  mem       REG              252,1    14608    1294585 /lib/libdl-2.9.so
    STAFProc  23126     evsadm  mem       REG              252,1    39112    1294584 /lib/libcrypt-2.9.so
    STAFProc  23126     evsadm  mem       REG              252,1   130151    1294596 /lib/libpthread-2.9.so
    STAFProc  23126     evsadm  mem       REG              252,1   135680    1294579 /lib/ld-2.9.so
    STAFProc  23126     evsadm  mem       REG              252,1   221604    1205793 /usr/lib/locale/en_US/LC_CTYPE
    STAFProc  23126     evsadm  mem       REG              252,1       59    1205794 /usr/lib/locale/en_US/LC_NUMERIC
    STAFProc  23126     evsadm  mem       REG              252,1     2459    1205795 /usr/lib/locale/en_US/LC_TIME
    STAFProc  23126     evsadm  mem       REG              252,1    19459    1205796 /usr/lib/locale/en_US/LC_COLLATE
    STAFProc  23126     evsadm  mem       REG              252,1      291    1205797 /usr/lib/locale/en_US/LC_MONETARY
    STAFProc  23126     evsadm  mem       REG              252,1       57    1205799 /usr/lib/locale/en_US/LC_MESSAGES/SYS_LC_MESSAGES
    STAFProc  23126     evsadm  mem       REG              252,1       39    1205800 /usr/lib/locale/en_US/LC_PAPER
    STAFProc  23126     evsadm  mem       REG              252,1       82    1205801 /usr/lib/locale/en_US/LC_NAME
    STAFProc  23126     evsadm  mem       REG              252,1   689115    1254852 /usr/local/staf/lib/libSTAF.so
    STAFProc  23126     evsadm  mem       REG              252,1      160    1205802 /usr/lib/locale/en_US/LC_ADDRESS
    STAFProc  23126     evsadm  mem       REG              252,1       64    1205803 /usr/lib/locale/en_US/LC_TELEPHONE
    STAFProc  23126     evsadm  mem       REG              252,1       28    1205804 /usr/lib/locale/en_US/LC_MEASUREMENT
    STAFProc  23126     evsadm  mem       REG              252,1      378    1205805 /usr/lib/locale/en_US/LC_IDENTIFICATION
    STAFProc  23126     evsadm    0r      CHR                1,3                3222 /dev/null
    STAFProc  23126     evsadm    1w      REG              252,1      132    2015236 /tmp/stafproc.out
    STAFProc  23126     evsadm    2w      REG              252,1      132    2015236 /tmp/stafproc.out
    STAFProc  23126     evsadm    3r      CHR                1,3                3222 /dev/null
    STAFProc  23126     evsadm    4w      REG              252,1      132    2015236 /tmp/stafproc.out
    STAFProc  23126     evsadm    5w      REG              252,1      132    2015236 /tmp/stafproc.out
    STAFProc  23126     evsadm    6uW     REG              252,1        0    2015237 /tmp/STAF.tmp
    STAFProc  23126     evsadm    7uW     REG              252,1        0    2015241 /tmp/DataDir_-usr-local-staf-data-STAF.tmp
    STAFProc  23126     evsadm    8u     unix 0xffff8800971f44c0          3654456439 /tmp/STAFIPC_STAF
    STAFProc  23126     evsadm   10u     IPv4         3654456442                 TCP *:6500 (LISTEN)
    STAFProc  23126     evsadm   11r      CHR                5,0                1696 /dev/tty
    STAFProc  23126     evsadm   12u     IPv4         3659156961                 TCP testmgr02.prod.evs:6500->scheduler01.prod.evs:52139 (ESTABLISHED)
    STAFProc  23126     evsadm   13u     unix 0xffff8800971f1e40          3654456510 /tmp/STAFIPC_STAF
    STAFProc  23126     evsadm   15u     unix 0xffff8801c5df7640          3755867973 /tmp/STAFIPC_STAF
    STAFProc  23126     evsadm   16u     IPv4         3659253886                 TCP testmgr02.prod.evs:6500->scheduler01.prod.evs:52675 (ESTABLISHED)
    STAFProc  23126     evsadm   17u     unix 0xffff8801ef484a40          3659253889 socket
    STAFProc  23126     evsadm   18u     unix 0xffff88009709cfc0          3659156964 socket
    STAFProc  23126     evsadm   21u     unix 0xffff8800971af900          3659156996 /tmp/STAFIPC_STAF
    STAFProc  23126     evsadm   22u     unix 0xffff88001cc4e5c0          3659253921 /tmp/STAFIPC_STAF
    sh        23146     evsadm  cwd       DIR              252,1     4096      32774 /home/evsadm
    sh        23146     evsadm  rtd       DIR              252,1     4096          2 /
    sh        23146     evsadm  txt       REG              252,1   105592    1146885 /bin/dash
    sh        23146     evsadm  mem       REG              252,1  1502512    1294582 /lib/libc-2.9.so
    sh        23146     evsadm  mem       REG              252,1   135680    1294579 /lib/ld-2.9.so
    sh        23146     evsadm    0r      CHR                1,3                3222 /dev/null
    sh        23146     evsadm    1w      REG              252,1   321690    1269873 /usr/local/staf/data/STAF/lang/java/jvm/STAFJVM1/JVMLog.1
    sh        23146     evsadm    2w      REG              252,1   321690    1269873 /usr/local/staf/data/STAF/lang/java/jvm/STAFJVM1/JVMLog.1
    sh        23146     evsadm    3r      CHR                1,3                3222 /dev/null
    sh        23146     evsadm    4w      REG              252,1      132    2015236 /tmp/stafproc.out
    sh        23146     evsadm    5w      REG              252,1      132    2015236 /tmp/stafproc.out
    sh        23146     evsadm    6u      REG              252,1        0    2015237 /tmp/STAF.tmp
    sh        23146     evsadm    7r      REG              252,1      827    1255132 /usr/local/staf/bin/STAF.cfg
    sh        23146     evsadm   11r      CHR                5,0                1696 /dev/tty
    java      23147     evsadm  cwd       DIR              252,1     4096      32774 /home/evsadm
    java      23147     evsadm  rtd       DIR              252,1     4096          2 /
    java      23147     evsadm  txt       REG              252,1    39504    1269786 /usr/lib/jvm/java-6-openjdk/jre/bin/java
    java      23147     evsadm  mem       REG              252,1    88688    1262353 /usr/lib/jvm/java-6-openjdk/jre/lib/amd64/libnet.so
    java      23147     evsadm  mem       REG              252,1   128713    1254850 /usr/local/staf/lib/libJSTAF.so
    java      23147     evsadm  mem       REG              252,1   640873    1269881 /usr/local/staf/data/STAF/lang/java/service/STAX/jars/jython.jar
    java      23147     evsadm  mem       REG              252,1   188993    1269880 /usr/local/staf/data/STAF/lang/java/service/STAX/jars/serializer.jar
    java      23147     evsadm  mem       REG              252,1  3078601    1269879 /usr/local/staf/data/STAF/lang/java/service/STAX/jars/xalan.jar
    java      23147     evsadm  mem       REG              252,1   124724    1269878 /usr/local/staf/data/STAF/lang/java/service/STAX/jars/xmlParserAPIs.jar
    java      23147     evsadm  mem       REG              252,1  1010806    1269877 /usr/local/staf/data/STAF/lang/java/service/STAX/jars/xercesImpl.jar
    java      23147     evsadm  mem       REG              252,1   509707    1254917 /usr/lib/jvm/java-6-openjdk/jre/lib/jsse.jar
    java      23147     evsadm  mem       REG              252,1  5718631    1269855 /usr/local/staf/services/stax/STAX.jar
    java      23147     evsadm  mem       REG              252,1    53726    1254847 /usr/local/staf/lib/libSTAFLIPC.so
    java      23147     evsadm  mem       REG              252,1       59    1205794 /usr/lib/locale/en_US/LC_NUMERIC
    java      23147     evsadm  mem       REG              252,1     2459    1205795 /usr/lib/locale/en_US/LC_TIME
    java      23147     evsadm  mem       REG              252,1    19459    1205796 /usr/lib/locale/en_US/LC_COLLATE
    java      23147     evsadm  mem       REG              252,1      291    1205797 /usr/lib/locale/en_US/LC_MONETARY
    java      23147     evsadm  mem       REG              252,1       57    1205799 /usr/lib/locale/en_US/LC_MESSAGES/SYS_LC_MESSAGES
    java      23147     evsadm  mem       REG              252,1       39    1205800 /usr/lib/locale/en_US/LC_PAPER
    java      23147     evsadm  mem       REG              252,1       82    1205801 /usr/lib/locale/en_US/LC_NAME
    java      23147     evsadm  mem       REG              252,1      160    1205802 /usr/lib/locale/en_US/LC_ADDRESS
    java      23147     evsadm  mem       REG              252,1       64    1205803 /usr/lib/locale/en_US/LC_TELEPHONE
    java      23147     evsadm  mem       REG              252,1    96560    1294400 /lib/libgcc_s.so.1
    java      23147     evsadm  mem       REG              252,1  1023448    1181022 /usr/lib/libstdc++.so.6.0.10
    java      23147     evsadm  mem       REG              252,1   689115    1254852 /usr/local/staf/lib/libSTAF.so
    java      23147     evsadm  mem       REG              252,1    39112    1294584 /lib/libcrypt-2.9.so
    java      23147     evsadm  mem       REG              252,1    69541    1254871 /usr/local/staf/lib/libJSTAFSH.so
    java      23147     evsadm  mem       REG              252,1   221604    1205793 /usr/lib/locale/en_US/LC_CTYPE
    java      23147     evsadm  mem       REG              252,1 61271271    1255113 /usr/lib/jvm/java-6-openjdk/jre/lib/rt.jar
    java      23147     evsadm  mem       REG              252,1       28    1205804 /usr/lib/locale/en_US/LC_MEASUREMENT
    java      23147     evsadm  mem       REG              252,1      378    1205805 /usr/lib/locale/en_US/LC_IDENTIFICATION
    java      23147     evsadm  mem       REG              252,1    32095    1269846 /usr/local/staf/services/stax/STAFEvent.jar
    java      23147     evsadm  mem       REG              252,1   448950    1222778 /usr/share/java/gnome-java-bridge.jar
    java      23147     evsadm  mem       REG              252,1    31064    1262360 /usr/lib/jvm/java-6-openjdk/jre/lib/amd64/libzip.so
    java      23147     evsadm  mem       REG              252,1    51616    1294591 /lib/libnss_files-2.9.so
    java      23147     evsadm  mem       REG              252,1    43472    1294593 /lib/libnss_nis-2.9.so
    java      23147     evsadm  mem       REG              252,1    35632    1294589 /lib/libnss_compat-2.9.so
    java      23147     evsadm  mem       REG              252,1    36432    1262362 /usr/lib/jvm/java-6-openjdk/jre/lib/amd64/native_threads/libhpi.so
    java      23147     evsadm  mem       REG              252,1    97168    1294588 /lib/libnsl-2.9.so
    java      23147     evsadm  mem       REG              252,1   197872    1262343 /usr/lib/jvm/java-6-openjdk/jre/lib/amd64/libjava.so
    java      23147     evsadm  mem       REG              252,1    67968    1262359 /usr/lib/jvm/java-6-openjdk/jre/lib/amd64/libverify.so
    java      23147     evsadm  mem       REG              252,1    31656    1294598 /lib/librt-2.9.so
    java      23147     evsadm  mem       REG              252,1   542928    1294586 /lib/libm-2.9.so
    java      23147     evsadm  mem       REG              252,1  8661576    1262364 /usr/lib/jvm/java-6-openjdk/jre/lib/amd64/server/libjvm.so
    java      23147     evsadm  mem       REG              252,1  1502512    1294582 /lib/libc-2.9.so
    java      23147     evsadm  mem       REG              252,1    14608    1294585 /lib/libdl-2.9.so
    java      23147     evsadm  mem       REG              252,1    18664    1262331 /usr/lib/jvm/java-6-openjdk/jre/lib/amd64/jli/libjli.so
    java      23147     evsadm  mem       REG              252,1   130151    1294596 /lib/libpthread-2.9.so
    java      23147     evsadm  mem       REG              252,1    96768    1294474 /lib/libz.so.1.2.3.3
    java      23147     evsadm  mem       REG              252,1   135680    1294579 /lib/ld-2.9.so
    java      23147     evsadm  mem       REG              252,1    70681    1254903 /usr/lib/jvm/java-6-openjdk/jre/lib/ext/pulse-java.jar
    java      23147     evsadm  mem       REG              252,1    32768    2015239 /tmp/hsperfdata_evsadm/23147
    java      23147     evsadm  mem       REG              252,1    89486    1254872 /usr/local/staf/lib/JSTAF.jar
    java      23147     evsadm    0r      CHR                1,3                3222 /dev/null
    java      23147     evsadm    1w      REG              252,1   321690    1269873 /usr/local/staf/data/STAF/lang/java/jvm/STAFJVM1/JVMLog.1
    java      23147     evsadm    2w      REG              252,1   321690    1269873 /usr/local/staf/data/STAF/lang/java/jvm/STAFJVM1/JVMLog.1
    java      23147     evsadm    3r      CHR                1,3                3222 /dev/null
    java      23147     evsadm    4w      REG              252,1      132    2015236 /tmp/stafproc.out
    java      23147     evsadm    5w      REG              252,1      132    2015236 /tmp/stafproc.out
    java      23147     evsadm    6u      REG              252,1        0    2015237 /tmp/STAF.tmp
    java      23147     evsadm    7r      REG              252,1      827    1255132 /usr/local/staf/bin/STAF.cfg
    java      23147     evsadm    8r      REG              252,1 61271271    1255113 /usr/lib/jvm/java-6-openjdk/jre/lib/rt.jar
    java      23147     evsadm    9r      REG              252,1    70681    1254903 /usr/lib/jvm/java-6-openjdk/jre/lib/ext/pulse-java.jar
    java      23147     evsadm   10r      REG              252,1   448950    1222778 /usr/share/java/gnome-java-bridge.jar
    java      23147     evsadm   11r      CHR                5,0                1696 /dev/tty
    java      23147     evsadm   12r      REG              252,1    89486    1254872 /usr/local/staf/lib/JSTAF.jar
    java      23147     evsadm   13r      CHR                1,3                3222 /dev/null
    java      23147     evsadm   14w      REG              252,1   321690    1269873 /usr/local/staf/data/STAF/lang/java/jvm/STAFJVM1/JVMLog.1
    java      23147     evsadm   15w      REG              252,1   321690    1269873 /usr/local/staf/data/STAF/lang/java/jvm/STAFJVM1/JVMLog.1
    java      23147     evsadm   16u     unix 0xffff8801418482c0          3654456431 /tmp/STAFIPC_STAFJSTAF_STAFJVM1
    java      23147     evsadm   17u     sock                0,4          4202140519 can’t identify protocol
    java      23147     evsadm   18u     unix 0xffff8800971f3440          3654456511 socket
    java      23147     evsadm   19r      REG              252,1  5718631    1269855 /usr/local/staf/services/stax/STAX.jar
    java      23147     evsadm   20r      REG              252,1   509707    1254917 /usr/lib/jvm/java-6-openjdk/jre/lib/jsse.jar
    java      23147     evsadm   21r      CHR                1,8                1534 /dev/random
    java      23147     evsadm   22r      CHR                1,9                1569 /dev/urandom
    java      23147     evsadm   23r      REG              252,1  1010806    1269877 /usr/local/staf/data/STAF/lang/java/service/STAX/jars/xercesImpl.jar
    java      23147     evsadm   24r      REG              252,1   124724    1269878 /usr/local/staf/data/STAF/lang/java/service/STAX/jars/xmlParserAPIs.jar
    java      23147     evsadm   25r      REG              252,1  3078601    1269879 /usr/local/staf/data/STAF/lang/java/service/STAX/jars/xalan.jar
    java      23147     evsadm   26r      REG              252,1   188993    1269880 /usr/local/staf/data/STAF/lang/java/service/STAX/jars/serializer.jar
    java      23147     evsadm   27r      REG              252,1   640873    1269881 /usr/local/staf/data/STAF/lang/java/service/STAX/jars/jython.jar
    java      23147     evsadm   28r      REG              252,1    32095    1269846 /usr/local/staf/services/stax/STAFEvent.jar
    java      23147     evsadm   29u     unix 0xffff88009709b700          3659156923 /tmp/STAFIPC_STAFJSTAF_STAFJVM1
    java      23147     evsadm   30u     unix 0xffff8800971a98c0          3659156998 socket
    java      23147     evsadm   31u     unix 0xffff8801ef487900          3659253742 /tmp/STAFIPC_STAFJSTAF_STAFJVM1
    java      23147     evsadm   32u     unix 0xffff88001cc4c200          3659253923 socket
    java      23147     evsadm   33u     unix 0xffff8801c5df5d80          3755867974 socket

    top — 12:02:25 up 25 days, 19:44,  2 users,  load average: 2.77, 2.72, 2.65
    Tasks: 124 total,   4 running, 120 sleeping,   0 stopped,   0 zombie
    Cpu(s): 30.7%us, 14.5%sy,  0.0%ni, 54.8%id,  0.0%wa,  0.0%hi,  0.1%si,  0.0%st
    Mem:   8067364k total,  7855960k used,   211404k free,   128772k buffers
    Swap:   999416k total,        0k used,   999416k free,  7172012k cached

      PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+ COMMAND                                                                                        
    25387 evsadm    18  -2 1197m 339m 9.8m S  106  4.3   1676:23 java                                                                                           
    25360 evsadm    18  -2  511m 9232 4420 S   74  0.1 917:19.32 STAFProc                                                                                       
    2778 zabbix    25   5 49824  984  564 S    0  0.0 343:05.87 zabbix_agentd

    evsadm@testmgr02:~$ staf local ping ping
    Error registering with STAF, RC: 22
    evsadm@testmgr02:~$ staf local ping ping
    Error registering with STAF, RC: 22
    evsadm@testmgr02:~$ staf local ping ping
    evsadm@testmgr02:~$ staf local ping ping
    evsadm@testmgr02:~$ staf local ping ping
    Error submitting request, RC: 22
    Additional info

    STAFConnectionReadUInt: Error reading from socket: recv() RC=104: 22
    evsadm@testmgr02:~$ staf local ping ping
    Error submitting request, RC: 22
    Additional info

    STAFConnectionReadUInt: Error reading from socket: recv() RC=104: 22

    $ tail /tmp/stafproc.out
    Machine          : testmgr02
    Machine nickname : testmgr02
    Startup time     : 20100312-11:55:08

    STAFProc version 3.3.4 initialized
    ssh_exchange_identification: Connection closed by remote host
    ssh_exchange_identification: Connection closed by remote host
    ssh_exchange_identification: Connection closed by remote host
    ssh_exchange_identification: Connection closed by remote host
    ssh_exchange_identification: Connection closed by remote host

  • Titus

    Got a similar problem, that is the same error:
    STAFConnectionWriteUInt: Error writing to socket: send() RC=104: 22

    The way i get this error if i use system variable retrieval simultaneously (through threads) for a remote server, so i’m not using local. I can replicate the behavior but for some servers it works ok. The only difference i found was STAFLoop takes 10 time longer to complete.

    Ping is ok:
    64 bytes from m250.company.com (192.168.160.250): icmp_seq=0 ttl=127 time=0.266 ms
    64 bytes from m250.company.com (192.168.160.250): icmp_seq=1 ttl=127 time=0.388 ms
    64 bytes from m250.company.com (192.168.160.250): icmp_seq=2 ttl=127 time=0.261 ms
    64 bytes from m250.company.com (192.168.160.250): icmp_seq=3 ttl=127 time=0.285 ms
    64 bytes from m250.company.com (192.168.160.250): icmp_seq=4 ttl=127 time=0.345 ms
    64 bytes from m250.company.com (192.168.160.250): icmp_seq=5 ttl=127 time=0.279 ms
    64 bytes from m250.company.com (192.168.160.250): icmp_seq=6 ttl=127 time=0.258 ms

    With loop ok:
    STAFLoop 10 192.168.160.250 ping ping
    Total loops: 10
    Total time : 4.391
    Avg. time  : 0.439
    Loops/sec  : 2.27739

    With low loops per sec:
    STAFLoop 10 192.168.160.250 ping ping
    Total loops: 10
    Total time : 47.807
    Avg. time  : 4.780
    Loops/sec  : 0.209174

    Tried increasing connection timeout and connection requests and got no improvement.
    All the servers are in the same vlan and ave same dns, network configs etc.
    Any help you may have, i’ll appreciate it.

    Thanks,
    Titus

  • Sharon Lucas

    You’re seeing error code 104 (ECONNRESET) on a socket send() function (the originator of this forum thread was seeing this error on a socket read() function.

    Linux socket error code 104 is defined as:

    #define ECONNRESET      104     /* Connection reset by peer */

    This error can occur in situations when the peer unexpectedly disappears (e.g. due to a crash, kill, reboot, Some firewall or router is sabotaging the connection, etc).  You can google for more information on this socket error code (ECONNRESET) that is returned by the socket send() function.

Log in to post a comment.

Источник

Понравилась статья? Поделить с друзьями:

Читайте также:

  • Socket processaccept error too many open files
  • Socket port bind error
  • Socket io unhandled error event
  • Socket io transport error
  • Socket init error

    • 0 0 голоса
    Рейтинг статьи
    Подписаться
    Уведомить о
    guest

    0 комментариев
    Старые
    Новые Популярные
    Межтекстовые Отзывы
    Посмотреть все комментарии