Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

503 Service Unavailable #206

Open
alexjfisher opened this issue Jun 21, 2017 · 2 comments
Open

503 Service Unavailable #206

alexjfisher opened this issue Jun 21, 2017 · 2 comments
Labels
bug

Comments

@alexjfisher
Copy link
Member

The code designed to retry api connections if Wildfly hasn't started up yet isn't working for me.

Error: /Stage[main]/Profile::Keycloak/Wildfly::Resource[/socket-binding-group=standard-sockets/socket-binding=proxy-https]/Wildfly_resource[/socket-binding-group=standard-sockets/socket-binding=proxy-https]: Could not evaluate: 757: unexpected token at '<html><head><title>Error</title></head><body>503 - Service Unavailable</body></html>'
/opt/puppetlabs/puppet/lib/ruby/2.1.0/json/common.rb:155:in `parse'
/opt/puppetlabs/puppet/lib/ruby/2.1.0/json/common.rb:155:in `parse'
/opt/puppetlabs/puppet/cache/lib/puppet_x/wildfly/api_client.rb:58:in `submit'
/opt/puppetlabs/puppet/cache/lib/puppet_x/wildfly/operation_request.rb:40:in `exists?'
/opt/puppetlabs/puppet/cache/lib/puppet/provider/wildfly_resource/http_api.rb:18:in `exists?'

It appears the @http_client.request authz_request get in authz_header works fine, but the post in submit gets a 503. The service is 'partially' started??
Furthermore, the 503 isn't returned as json so I get the unexpected token exception instead of the "Failed with: #{response['failure-description']} for #{body.to_json}" one.

I'm using the keycloak distribution on CentOS 7 with puppet 4.10.3. I think you should be able to reproduce the issue with the following and puppet apply.

class { '::wildfly':
    version           => '10.1.0',
    distribution      => 'wildfly',
    config            => 'standalone-ha.xml',
    install_cache_dir => '/tmp',
    install_source    => 'https://downloads.jboss.org/keycloak/3.1.0.Final/keycloak-3.1.0.Final.tar.gz',
    mgmt_user         => {
      username => 'admin',
      password => 'admin',
    },
  }
wildfly::resource { '/socket-binding-group=standard-sockets/socket-binding=proxy-https':
    content => {
      port => 443,
    },
    notify  => Wildfly::Reload['Reload if necessary'],
}

wildfly::reload { 'Reload if necessary':
    retries => 2,
    wait    => 15,
}

Many thanks,
Alex
@cfrantsen
Copy link
Contributor

I have seen this as well using the standard wildfly distribution, so it's not related to keycloak. Could perhaps be solved by checking if response is an instance of Net::HTTPSuccess, and if not trigger a retry?

@jairojunior
Copy link
Collaborator

Just confirmed this. Don't even need wildfly::reload to reproduce it.

It's not happening during retry - and the retried request needs to return 401 (Net::HTTPUnauthorized) with data to perform digest authentication.

This is happening afterwards during the actual Management Request - as far as I understood, Management API is ready to process authentication, but Wildfly is not ready to receive any management operations, hence the 503.

Sysvinit scripts used to work by checking Wildfly's logs for JBAS015874, but they replaced it with a markerfile approach. Bundled systemd scripts does not use any of this.

I need another way to assert that Wildfly is ready cause I can't retry actual management requests since some of them are not idempotent. It was OK to retry authentication.

One alternative would be to not use bundled systemd script but systemd/sysvinit wrapper (like Wildfly < 10: https://github.com/biemond/biemond-wildfly/blob/master/templates/wildfly.sysvinit.service.epp) or create a new script.

@vStone vStone mentioned this issue Feb 12, 2018
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@alexjfisher @jairojunior @cfrantsen