https url in blocklists "cannot assign requested address"

[aww+]

http urls work just fine in the csf.blocklists but not https

CentOS6 but all the proper libraries for Perl SSL have been installed

"cannot assign requested address"

not the normal error but a binding error for the local IP address - what would cause this?

doing a wget for the https url from the command line works just fine

so either CSF is seeing the local server IP address correctly or somehow another service like nginx is blocking it?

[ForumAdmin]

It appears from investigation that this might be to do with some kind of limit being imposed on the server for outbound SSL connections for some reason (an external firewall perhaps).

The solution seems to be to reuse an existing connection in LWP. You could try the following:

edit /usr/local/csf/lib/ConfigServer/URLGet.pm and find line 143:

Code: Select all

	$ua->timeout(30);

after it add the following lines:

Code: Select all

use LWP::ConnCache;
my $cache = LWP::ConnCache->new;
$cache->total_capacity([1]);
$ua->conn_cache($cache);

Then restart csf and then lfd.
If that works, then we can look into it further.

[aww+]

I appreciate the effort to research and come up with that code, sadly it didn't change the behavior

exact https url can be fetched without problem from the server by
wget
curl (command line)
php via file_get_contents($url)
php via curl library

can you give me minimal test code in perl to see if it is the perl library in general that is failing somehow?

I mean just a few lines to see if the url can be fetched

edit: I came up with this test code after googling a bit and it works, but CSF/LFD probably works differently

Code: Select all

#!/usr/bin/perl
use Net::SSL (); 
use LWP::UserAgent;
$Net::HTTPS::SSL_SOCKET_CLASS = "Net::SSL"; 
$ENV{PERL_LWP_SSL_VERIFY_HOSTNAME} = 0;
my $ua = LWP::UserAgent->new();
my $req = HTTP::Request->new('GET','https://www.example-url-goes-here');
my $res = $ua->request($req);
print $res->content;

[ForumAdmin]

The code posted for for combating issues with limitations on fast repeating outbound SSL connections. For example:

Code: Select all

removed in preference to below

To compare with the other methods, you need to loop them as above.

It might help for you to post the complete error line that you are seeing.

[ForumAdmin]

This is actually a better representation of what happens:

Code: Select all

#!/usr/bin/perl
use strict;
use LWP::UserAgent;

my $url = 'https://www.google.com';

$| = 1;
for (1..100) {
	my $ua = LWP::UserAgent->new;
	my $response = $ua->get($url);
	if ($response->is_success) {print "."} else {print "Error: ".$response->status_line."\n"}
}
print "done.\n";

[aww+]

This is actually a better representation of what happens:

that simple test does fail with that exact "assigned address" error, so this has to be something very basic and not CSF/LFD's fault

maybe I have a bad library install or something is outdated or there is a bad rule in the firewall somewhere

Code: Select all

*******************************************************************
 Using the default of SSL_verify_mode of SSL_VERIFY_NONE for client
 is deprecated! Please set SSL_verify_mode to SSL_VERIFY_PEER
 together with SSL_ca_file|SSL_ca_path for verification.
 If you really don't want to verify the certificate and keep the
 connection open to Man-In-The-Middle attacks please set
 SSL_verify_mode explicitly to SSL_VERIFY_NONE in your application.
*******************************************************************
  at /usr/share/perl5/vendor_perl/LWP/Protocol/http.pm line 31
Error: 500 Can't connect to www.google.com:443 (connect: Cannot assign requested address)
done.

oh and same result without the ssl error when I set $ua->ssl_opts(SSL_verify_mode=>'0');

Code: Select all

Error: 500 Can't connect to www.google.com:443 (connect: Cannot assign requested address)

server only has a single ip address with a single route setup so really that can't be the issue

and again wget, curl, php get all work

so this has to be my perl configuration I guess

or the LWP library is very intolerant

current theory: LWP is looking up the local IP address to use in SSL from some configuration file in linux somewhere that is reporting it wrong - the version of LWP in centos6 is 5,833 which is rather old, though they do backport security fixes they often do not backport bug fixes

[aww+]

this solves it, I am not sure why LWP is grabbing the wrong local address only for SSL

Code: Select all

push(@LWP::Protocol::http::EXTRA_SOCK_OPTS, "LocalAddr" => "1.2.3.4" );

where 1.2.3.4 is the local machine internet address of course

the version of LWP on centos6 base is too old to support the LWP "local_address" function so that is the old legacy method

somewhere in this machine configuration it has the wrong address or maybe even 127.0.0.1 is being grabbed by LWP by default

after a bunch and bunch of reading I think this goes back to INADDR_ANY and I am still not sure how it discovers available local addresses which obviously is being picked incorrectly