Page 1 of 1
https url in blocklists "cannot assign requested address"
Posted: 16 Sep 2017, 18:39
by aww+
http urls work just fine in the csf.blocklists but not https
CentOS6 but all the proper libraries for Perl SSL have been installed
"cannot assign requested address"
not the normal error but a binding error for the local IP address - what would cause this?
doing a wget for the https url from the command line works just fine
so either CSF is seeing the local server IP address correctly or somehow another service like nginx is blocking it?
Re: https url in blocklists "cannot assign requested address"
Posted: 16 Sep 2017, 21:07
by ForumAdmin
It appears from investigation that this might be to do with some kind of limit being imposed on the server for outbound SSL connections for some reason (an external firewall perhaps).
The solution seems to be to reuse an existing connection in LWP. You could try the following:
edit /usr/local/csf/lib/ConfigServer/URLGet.pm and find line 143:
after it add the following lines:
Code: Select all
use LWP::ConnCache;
my $cache = LWP::ConnCache->new;
$cache->total_capacity([1]);
$ua->conn_cache($cache);
Then restart csf and then lfd.
If that works, then we can look into it further.
Re: https url in blocklists "cannot assign requested address"
Posted: 18 Sep 2017, 15:16
by aww+
I appreciate the effort to research and come up with that code, sadly it didn't change the behavior
exact https url can be fetched without problem from the server by
wget
curl (command line)
php via file_get_contents($url)
php via curl library
can you give me minimal test code in perl to see if it is the perl library in general that is failing somehow?
I mean just a few lines to see if the url can be fetched
edit: I came up with this test code after googling a bit and it works, but CSF/LFD probably works differently
Code: Select all
#!/usr/bin/perl
use Net::SSL ();
use LWP::UserAgent;
$Net::HTTPS::SSL_SOCKET_CLASS = "Net::SSL";
$ENV{PERL_LWP_SSL_VERIFY_HOSTNAME} = 0;
my $ua = LWP::UserAgent->new();
my $req = HTTP::Request->new('GET','https://www.example-url-goes-here');
my $res = $ua->request($req);
print $res->content;
Re: https url in blocklists "cannot assign requested address"
Posted: 18 Sep 2017, 16:12
by ForumAdmin
The code posted for for combating issues with limitations on fast repeating outbound SSL connections. For example:
To compare with the other methods, you need to loop them as above.
It might help for you to post the complete error line that you are seeing.
Re: https url in blocklists "cannot assign requested address"
Posted: 18 Sep 2017, 16:16
by ForumAdmin
This is actually a better representation of what happens:
Code: Select all
#!/usr/bin/perl
use strict;
use LWP::UserAgent;
my $url = 'https://www.google.com';
$| = 1;
for (1..100) {
my $ua = LWP::UserAgent->new;
my $response = $ua->get($url);
if ($response->is_success) {print "."} else {print "Error: ".$response->status_line."\n"}
}
print "done.\n";
Re: https url in blocklists "cannot assign requested address"
Posted: 19 Sep 2017, 17:58
by aww+
ForumAdmin wrote: ↑18 Sep 2017, 16:16
This is actually a better representation of what happens:
that simple test does fail with that exact "assigned address" error, so this has to be something very basic and not CSF/LFD's fault
maybe I have a bad library install or something is outdated or there is a bad rule in the firewall somewhere
Code: Select all
*******************************************************************
Using the default of SSL_verify_mode of SSL_VERIFY_NONE for client
is deprecated! Please set SSL_verify_mode to SSL_VERIFY_PEER
together with SSL_ca_file|SSL_ca_path for verification.
If you really don't want to verify the certificate and keep the
connection open to Man-In-The-Middle attacks please set
SSL_verify_mode explicitly to SSL_VERIFY_NONE in your application.
*******************************************************************
at /usr/share/perl5/vendor_perl/LWP/Protocol/http.pm line 31
Error: 500 Can't connect to www.google.com:443 (connect: Cannot assign requested address)
done.
oh and same result without the ssl error when I set $ua->ssl_opts(SSL_verify_mode=>'0');
Code: Select all
Error: 500 Can't connect to www.google.com:443 (connect: Cannot assign requested address)
server only has a single ip address with a single route setup so really that can't be the issue
and again wget, curl, php get all work
so this has to be my perl configuration I guess
or the LWP library is very intolerant
current theory: LWP is looking up the local IP address to use in SSL from some configuration file in linux somewhere that is reporting it wrong - the version of LWP in centos6 is 5,833 which is rather old, though they do backport security fixes they often do not backport bug fixes
Re: https url in blocklists "cannot assign requested address"
Posted: 19 Sep 2017, 18:21
by aww+
this solves it, I am not sure why LWP is grabbing the wrong local address only for SSL
Code: Select all
push(@LWP::Protocol::http::EXTRA_SOCK_OPTS, "LocalAddr" => "1.2.3.4" );
where 1.2.3.4 is the local machine internet address of course
the version of LWP on centos6 base is too old to support the LWP "local_address" function so that is the old legacy method
somewhere in this machine configuration it has the wrong address or maybe even 127.0.0.1 is being grabbed by LWP by default
after a bunch and bunch of reading I think this goes back to INADDR_ANY and I am still not sure how it discovers available local addresses which obviously is being picked incorrectly