diff --git a/docs/content/docs/deployment/advanced/historyserver.md b/docs/content/docs/deployment/advanced/historyserver.md index 7d0053bb23025..436cdab73b6f8 100644 --- a/docs/content/docs/deployment/advanced/historyserver.md +++ b/docs/content/docs/deployment/advanced/historyserver.md @@ -43,6 +43,7 @@ bin/historyserver.sh (start|start-foreground|stop) ``` By default, this server binds to `localhost` and listens at port `8082`. +If you bind the HistoryServer to a network-facing address such as `0.0.0.0`, enable authentication or place it behind an authenticated gateway. Without authentication, any user with network access can inspect archived job metadata, execution plans, configuration values, timestamps, and configured log links. Currently, you can only run it as a standalone process. @@ -75,6 +76,33 @@ The contained archives are downloaded and cached in the local filesystem. The lo Check out the configuration page for a [complete list of configuration options]({{< ref "docs/deployment/config" >}}#history-server). +## SPNEGO Authentication + +The HistoryServer web UI and REST endpoints can require Kerberos/SPNEGO authentication. This is opt-in; the default `historyserver.web.authentication.type` is `NONE`. + +To enable SPNEGO, configure an HTTP service principal and keytab for the HistoryServer process: + +```yaml +historyserver.web.authentication.type: KERBEROS +historyserver.web.authentication.kerberos.principal: HTTP/_HOST@EXAMPLE.COM +historyserver.web.authentication.kerberos.keytab: /etc/security/keytabs/flink-historyserver.keytab +historyserver.web.authentication.signature.secret-file: /etc/flink/historyserver-auth-secret +``` + +The principal must start with `HTTP/`. The `_HOST` placeholder is replaced with the local hostname, and `*` can be used to accept all `HTTP/` principals present in the configured keytab. If `historyserver.web.authentication.kerberos.name-rules` is not configured, the default Hadoop auth-to-local rule is used. + +When authentication is enabled, requests without a valid cookie or `Authorization: Negotiate` header return `401` with `WWW-Authenticate: Negotiate`. A valid SPNEGO exchange issues a signed `hadoop.auth` cookie and then proceeds to the existing HistoryServer handlers. Invalid SPNEGO tokens return `403`. + +Clients with Kerberos credentials can access the HistoryServer with: + +```shell +curl --negotiate -u : http://historyserver.example.com:8082/jobs/overview +``` + +If multiple HistoryServer instances serve the same endpoint, configure the same `historyserver.web.authentication.signature.secret` or `historyserver.web.authentication.signature.secret-file` on all instances so that authentication cookies are accepted consistently. If neither option is configured, a random process-local signing secret is generated at startup. + +SPNEGO authenticates the caller only. It does not add per-user or per-group authorization, and it does not change JobManager or TaskManager REST authentication. + ## Log Integration Flink does not provide built-in methods for archiving logs of completed jobs. diff --git a/docs/content/docs/deployment/config.md b/docs/content/docs/deployment/config.md index 4b1d24bc1f2b1..ff7e145d55c88 100644 --- a/docs/content/docs/deployment/config.md +++ b/docs/content/docs/deployment/config.md @@ -161,11 +161,14 @@ You can configure checkpointing directly in code within your Flink job or applic **Web UI** + - `web.authentication.type`: Controls authentication for the JobManager Web UI and REST endpoint *(NONE by default)*. Set this to `KERBEROS` to require SPNEGO authentication on all `8081` paths. - `web.submit.enable`: Enables uploading and starting jobs through the Flink UI *(true by default)*. Please note that even when this is disabled, session clusters still accept jobs through REST requests (HTTP calls). This flag only guards the feature to upload jobs in the UI. - `web.cancel.enable`: Enables canceling jobs through the Flink UI *(true by default)*. Please note that even when this is disabled, session clusters still cancel jobs through REST requests (HTTP calls). This flag only guards the feature to cancel jobs in the UI. - `web.upload.dir`: The directory where to store uploaded jobs. Only used when `web.submit.enable` is true. - `web.exception-history-size`: Sets the size of the exception history that prints the most recent failures that were handled by Flink for a job. +Exposing the JobManager Web UI / REST endpoint to an untrusted network without authentication allows unauthenticated job submission and control operations. Enable `web.authentication.type: KERBEROS` or place the endpoint behind an authenticated proxy before exposing port `8081`. + **Other** - `io.tmp.dirs`: The directories where Flink puts local data, defaults to the system temp directory (`java.io.tmpdir` property). If a list of directories is configured, Flink will rotate files across the directories. diff --git a/docs/content/docs/deployment/security/security-kerberos.md b/docs/content/docs/deployment/security/security-kerberos.md index 8d7d3bfd55c30..db683325f1032 100644 --- a/docs/content/docs/deployment/security/security-kerberos.md +++ b/docs/content/docs/deployment/security/security-kerberos.md @@ -60,6 +60,7 @@ The following services and connectors are supported for Kerberos authentication: - HDFS - HBase - ZooKeeper +- JobManager Web UI / REST endpoint (SPNEGO) Note that it is possible to enable the use of Kerberos independently for each service or connector. For example, the user may enable Hadoop security without necessitating the use of Kerberos for ZooKeeper, @@ -92,6 +93,34 @@ dynamic entries provided by this module. This module configures certain process-wide ZooKeeper security-related settings, namely the ZooKeeper service name (default: `zookeeper`) and the JAAS login context name (default: `Client`). +### JobManager Web UI SPNEGO Authentication + +The JobManager Web UI and REST endpoint are unauthenticated by default. If port `8081` is reachable from an untrusted network, unauthenticated users can submit jobs and perform control operations. To require Kerberos/SPNEGO authentication on all JobManager WebMonitor paths, enable: + +```yaml +web.authentication.type: KERBEROS +web.authentication.kerberos.principal: HTTP/_HOST@EXAMPLE.COM +web.authentication.kerberos.keytab: /etc/security/keytabs/flink-jobmanager-http.keytab +web.authentication.kerberos.name-rules: DEFAULT +web.authentication.token.validity: 10 h +web.authentication.cookie.path: / +web.authentication.signature.secret-file: /etc/flink/jobmanager-web-auth-secret +``` + +The principal must start with `HTTP/`. The `_HOST` placeholder is replaced from `rest.address`, then `rest.bind-address`, then the local canonical hostname. Use `*` to accept all `HTTP/` principals present in the configured keytab. + +When enabled, requests without a valid authentication cookie or `Authorization: Negotiate` header receive `401` with `WWW-Authenticate: Negotiate`; malformed or invalid SPNEGO tokens receive `403`. A successful SPNEGO exchange issues a signed `hadoop.auth` cookie, so subsequent requests do not need to negotiate Kerberos again until the cookie expires. + +Clients with Kerberos credentials can access the endpoint with: + +```bash +curl --negotiate -u : http://jobmanager.example.com:8081/jobs +``` + +For multiple JobManager instances behind the same address, configure a shared `web.authentication.signature.secret` or `web.authentication.signature.secret-file`; otherwise each process uses a random local signing secret and cannot verify cookies issued by another process. + +SPNEGO authenticates the caller only. It does not add per-user authorization, user or group allowlists, Ranger policy enforcement, Knox integration, session ownership enforcement, or built-in Flink CLI / REST client SPNEGO support. + ## Deployment Modes Here is some information specific to each deployment mode. diff --git a/docs/content/docs/dev/table/sql-gateway/overview.md b/docs/content/docs/dev/table/sql-gateway/overview.md index 0a40983dbaee7..a44a72f37c3b2 100644 --- a/docs/content/docs/dev/table/sql-gateway/overview.md +++ b/docs/content/docs/dev/table/sql-gateway/overview.md @@ -57,6 +57,10 @@ $ ./bin/sql-gateway.sh start -Dsql-gateway.endpoint.rest.address=localhost The command starts the SQL Gateway with REST Endpoint that listens on the address localhost:8083. You can use the curl command to check whether the REST Endpoint is available. +{{< hint warning >}} +The SQL Gateway REST endpoint does not authenticate requests by default. If it is bound to a network-facing address, any client with network access can create sessions and execute SQL unless you enable REST SPNEGO authentication or protect the endpoint with external access controls. +{{< /hint >}} + ```bash $ curl http://localhost:8083/v1/info {"productName":"Apache Flink","version":"{{< version >}}"} diff --git a/docs/content/docs/dev/table/sql-gateway/rest.md b/docs/content/docs/dev/table/sql-gateway/rest.md index 40d109abbbbd1..e9a8902211ab6 100644 --- a/docs/content/docs/dev/table/sql-gateway/rest.md +++ b/docs/content/docs/dev/table/sql-gateway/rest.md @@ -88,9 +88,85 @@ Endpoint Options Integer The port that the client connects to. If bind-port has not been specified, then the sql gateway server will bind to this port. + +
sql-gateway.endpoint.rest.authentication.type
+ NONE + Enum + Authentication type for the SQL Gateway REST endpoint. Supported values are NONE and KERBEROS. + + +
sql-gateway.endpoint.rest.authentication.kerberos.principal
+ (none) + String + Kerberos principal for accepting SPNEGO requests when REST authentication type is KERBEROS. The value supports HTTP/_HOST@REALM host replacement and * to load all HTTP principals from the keytab. + + +
sql-gateway.endpoint.rest.authentication.kerberos.keytab
+ (none) + String + Path to the keytab containing the REST SPNEGO service principal. This option is required when REST authentication type is KERBEROS. + + +
sql-gateway.endpoint.rest.authentication.kerberos.name-rules
+ DEFAULT + String + Kerberos auth-to-local rules used to map the authenticated Kerberos principal to a local user name. + + +
sql-gateway.endpoint.rest.authentication.token.validity
+ 10 h + Duration + Validity of the signed REST authentication cookie issued after a successful SPNEGO exchange. + + +
sql-gateway.endpoint.rest.authentication.cookie.path
+ / + String + Path attribute for the REST authentication cookie. + + +
sql-gateway.endpoint.rest.authentication.signature.secret
+ (none) + String + Shared secret used to sign REST authentication cookies. If neither this option nor signature.secret-file is configured, a random process-local secret is generated. + + +
sql-gateway.endpoint.rest.authentication.signature.secret-file
+ (none) + String + Path to a file containing the shared secret used to sign REST authentication cookies. Configure either signature.secret or signature.secret-file, but not both. + +REST SPNEGO Authentication +---------------- + +By default, the REST endpoint does not authenticate requests. If the endpoint is reachable from an untrusted network, any client that can connect to it can create sessions, execute SQL, submit jobs, and access configured data sources. Bind the endpoint to a trusted interface, place it behind an authenticated gateway, or enable SPNEGO authentication before exposing it beyond a trusted boundary. + +SPNEGO authentication is opt-in and applies to every SQL Gateway REST path, including `/v*/info`. + +```yaml +sql-gateway.endpoint.rest.authentication.type: KERBEROS +sql-gateway.endpoint.rest.authentication.kerberos.principal: HTTP/_HOST@EXAMPLE.COM +sql-gateway.endpoint.rest.authentication.kerberos.keytab: /etc/security/keytabs/flink-sql-gateway.keytab +sql-gateway.endpoint.rest.authentication.kerberos.name-rules: DEFAULT +sql-gateway.endpoint.rest.authentication.signature.secret-file: /etc/flink/sql-gateway-cookie-secret +``` + +Clients must use an HTTP client that can negotiate SPNEGO, for example: + +```bash +$ kinit user@EXAMPLE.COM +$ curl --negotiate -u : http://sql-gateway-host:8083/v1/info +$ curl --negotiate -u : --request POST http://sql-gateway-host:8083/v1/sessions +``` + +Unauthenticated requests receive `401` with `WWW-Authenticate: Negotiate`. Malformed or invalid SPNEGO tokens receive `403`. +After a successful SPNEGO exchange, the REST endpoint issues a signed `hadoop.auth` cookie. Multi-instance or load-balanced SQL Gateway deployments must configure the same `sql-gateway.endpoint.rest.authentication.signature.secret` or `sql-gateway.endpoint.rest.authentication.signature.secret-file` on every instance; otherwise cookies issued by one instance will not be accepted by another. + +The built-in Flink SQL Client and Flink JDBC Driver do not negotiate SPNEGO with the SQL Gateway REST endpoint in this change. Use an HTTP SPNEGO-capable client or put the SQL Gateway behind a separate authenticated gateway or proxy when those clients are required. + REST API ---------------- diff --git a/docs/layouts/shortcodes/generated/history_server_configuration.html b/docs/layouts/shortcodes/generated/history_server_configuration.html index 8b171ba627c41..01d12a0c5fe06 100644 --- a/docs/layouts/shortcodes/generated/history_server_configuration.html +++ b/docs/layouts/shortcodes/generated/history_server_configuration.html @@ -56,6 +56,54 @@ String Address of the HistoryServer's web interface. + +
historyserver.web.authentication.cookie.path
+ "/" + String + Cookie path for HistoryServer SPNEGO authentication cookies. + + +
historyserver.web.authentication.kerberos.keytab
+ (none) + String + Absolute path to the keytab file for the HistoryServer SPNEGO principal. Required when HistoryServer web authentication type is KERBEROS. + + +
historyserver.web.authentication.kerberos.name-rules
+ (none) + String + Kerberos auth-to-local rules used to map authenticated principals to local user names. If unset, the default rule is used. + + +
historyserver.web.authentication.kerberos.principal
+ (none) + String + Kerberos principal for HistoryServer SPNEGO authentication. Required when HistoryServer web authentication type is KERBEROS. The principal must start with HTTP/. The _HOST placeholder is replaced with the local hostname. Use * to accept all HTTP principals in the keytab. + + +
historyserver.web.authentication.signature.secret
+ (none) + String + Shared secret for signing HistoryServer SPNEGO authentication cookies. If neither this option nor the secret-file option is set, a random process-local secret is generated. Configure the same secret for all HistoryServer instances that should accept each other's authentication cookies. + + +
historyserver.web.authentication.signature.secret-file
+ (none) + String + File containing the shared secret for signing HistoryServer SPNEGO authentication cookies. Configure the same secret file for all HistoryServer instances that should accept each other's authentication cookies. + + +
historyserver.web.authentication.token.validity
+ 10 h + Duration + Validity period for HistoryServer SPNEGO authentication cookies. + + +
historyserver.web.authentication.type
+ NONE +

Enum

+ Authentication type for the HistoryServer web frontend and REST endpoints.

Possible values: +
historyserver.web.port
8082 diff --git a/docs/layouts/shortcodes/generated/sql_gateway_rest_configuration.html b/docs/layouts/shortcodes/generated/sql_gateway_rest_configuration.html index 96808c286f53a..3f0ed3f2acc0c 100644 --- a/docs/layouts/shortcodes/generated/sql_gateway_rest_configuration.html +++ b/docs/layouts/shortcodes/generated/sql_gateway_rest_configuration.html @@ -14,6 +14,54 @@ String The address that should be used by clients to connect to the sql gateway server. + +
authentication.cookie.path
+ "/" + String + Cookie path used for SQL Gateway REST authentication cookies. + + +
authentication.kerberos.keytab
+ (none) + String + Kerberos keytab used by the SQL Gateway REST endpoint SPNEGO acceptor. + + +
authentication.kerberos.name-rules
+ "DEFAULT" + String + Kerberos auth-to-local rules used to derive the local user name from the authenticated Kerberos principal. + + +
authentication.kerberos.principal
+ (none) + String + Kerberos principal used by the SQL Gateway REST endpoint SPNEGO acceptor. The HTTP/_HOST@REALM pattern is supported, and '*' uses all HTTP principals from the keytab. + + +
authentication.signature.secret
+ (none) + String + Shared secret used to sign SQL Gateway REST authentication cookies. If neither this option nor authentication.signature.secret-file is configured, a random per-process secret is used. + + +
authentication.signature.secret-file
+ (none) + String + File containing the shared secret used to sign SQL Gateway REST authentication cookies. If neither this option nor authentication.signature.secret is configured, a random per-process secret is used. + + +
authentication.token.validity
+ 10 h + Duration + Validity period of the SQL Gateway REST authentication cookie. + + +
authentication.type
+ NONE +

Enum

+ The authentication type for the SQL Gateway REST endpoint. Supported values are NONE and KERBEROS.

Possible values: +
bind-address
(none) diff --git a/docs/layouts/shortcodes/generated/web_configuration.html b/docs/layouts/shortcodes/generated/web_configuration.html index faeb45e392d7c..016dcb72eaa16 100644 --- a/docs/layouts/shortcodes/generated/web_configuration.html +++ b/docs/layouts/shortcodes/generated/web_configuration.html @@ -14,6 +14,54 @@ String Access-Control-Allow-Origin header for all responses from the web-frontend. + +
web.authentication.cookie.path
+ "/" + String + Cookie path for JobManager web SPNEGO authentication cookies. + + +
web.authentication.kerberos.keytab
+ (none) + String + Absolute path to the keytab file for the JobManager web SPNEGO principal. Required when JobManager web authentication type is KERBEROS. + + +
web.authentication.kerberos.name-rules
+ "DEFAULT" + String + Kerberos auth-to-local rules used to map authenticated principals to local user names. + + +
web.authentication.kerberos.principal
+ (none) + String + Kerberos principal for JobManager web SPNEGO authentication. Required when JobManager web authentication type is KERBEROS. The principal must start with HTTP/. The _HOST placeholder is replaced with the configured REST address. Use * to accept all HTTP principals in the keytab. + + +
web.authentication.signature.secret
+ (none) + String + Shared secret for signing JobManager web SPNEGO authentication cookies. If neither this option nor the secret-file option is set, a random process-local secret is generated. Configure the same secret for all JobManager instances that should accept each other's authentication cookies. + + +
web.authentication.signature.secret-file
+ (none) + String + File containing the shared secret for signing JobManager web SPNEGO authentication cookies. Configure the same secret file for all JobManager instances that should accept each other's authentication cookies. + + +
web.authentication.token.validity
+ 10 h + Duration + Validity period for JobManager web SPNEGO authentication cookies. + + +
web.authentication.type
+ NONE +

Enum

+ Authentication type for the JobManager web frontend and REST endpoints.

Possible values: +
web.cancel.enable
true diff --git a/flink-core/src/main/java/org/apache/flink/configuration/HistoryServerOptions.java b/flink-core/src/main/java/org/apache/flink/configuration/HistoryServerOptions.java index 1628d266f3516..e6c860644bf7b 100644 --- a/flink-core/src/main/java/org/apache/flink/configuration/HistoryServerOptions.java +++ b/flink-core/src/main/java/org/apache/flink/configuration/HistoryServerOptions.java @@ -127,6 +127,88 @@ public class HistoryServerOptions { "Enable HTTPs access to the HistoryServer web frontend. This is applicable only when the" + " global SSL flag security.ssl.enabled is set to true."); + /** Authentication type for the HistoryServer web-frontend. */ + public static final ConfigOption + HISTORY_SERVER_WEB_AUTHENTICATION_TYPE = + key("historyserver.web.authentication.type") + .enumType(HistoryServerWebAuthenticationType.class) + .defaultValue(HistoryServerWebAuthenticationType.NONE) + .withDescription( + "Authentication type for the HistoryServer web frontend and " + + "REST endpoints."); + + /** Kerberos principal for HistoryServer SPNEGO authentication. */ + public static final ConfigOption HISTORY_SERVER_WEB_AUTHENTICATION_KERBEROS_PRINCIPAL = + key("historyserver.web.authentication.kerberos.principal") + .stringType() + .noDefaultValue() + .withDescription( + "Kerberos principal for HistoryServer SPNEGO authentication. " + + "Required when HistoryServer web authentication type " + + "is KERBEROS. The principal must start with HTTP/. " + + "The _HOST placeholder is replaced with the local " + + "hostname. Use * to accept all HTTP principals in the " + + "keytab."); + + /** Kerberos keytab for HistoryServer SPNEGO authentication. */ + public static final ConfigOption HISTORY_SERVER_WEB_AUTHENTICATION_KERBEROS_KEYTAB = + key("historyserver.web.authentication.kerberos.keytab") + .stringType() + .noDefaultValue() + .withDescription( + "Absolute path to the keytab file for the HistoryServer SPNEGO " + + "principal. Required when HistoryServer web authentication " + + "type is KERBEROS."); + + /** Kerberos auth-to-local rules for HistoryServer SPNEGO authentication. */ + public static final ConfigOption HISTORY_SERVER_WEB_AUTHENTICATION_KERBEROS_NAME_RULES = + key("historyserver.web.authentication.kerberos.name-rules") + .stringType() + .noDefaultValue() + .withDescription( + "Kerberos auth-to-local rules used to map authenticated principals to " + + "local user names. If unset, the default rule is used."); + + /** Validity of HistoryServer web authentication tokens. */ + public static final ConfigOption HISTORY_SERVER_WEB_AUTHENTICATION_TOKEN_VALIDITY = + key("historyserver.web.authentication.token.validity") + .durationType() + .defaultValue(Duration.ofHours(10)) + .withDescription( + "Validity period for HistoryServer SPNEGO authentication cookies."); + + /** Cookie path for HistoryServer web authentication tokens. */ + public static final ConfigOption HISTORY_SERVER_WEB_AUTHENTICATION_COOKIE_PATH = + key("historyserver.web.authentication.cookie.path") + .stringType() + .defaultValue("/") + .withDescription( + "Cookie path for HistoryServer SPNEGO authentication cookies."); + + /** Shared signing secret for HistoryServer web authentication tokens. */ + public static final ConfigOption HISTORY_SERVER_WEB_AUTHENTICATION_SIGNATURE_SECRET = + key("historyserver.web.authentication.signature.secret") + .stringType() + .noDefaultValue() + .withDescription( + "Shared secret for signing HistoryServer SPNEGO authentication " + + "cookies. If neither this option nor the secret-file option " + + "is set, a random process-local secret is generated. " + + "Configure the same secret for all HistoryServer instances " + + "that should accept each other's authentication cookies."); + + /** File containing the shared signing secret for HistoryServer web authentication tokens. */ + public static final ConfigOption + HISTORY_SERVER_WEB_AUTHENTICATION_SIGNATURE_SECRET_FILE = + key("historyserver.web.authentication.signature.secret-file") + .stringType() + .noDefaultValue() + .withDescription( + "File containing the shared secret for signing HistoryServer " + + "SPNEGO authentication cookies. Configure the same " + + "secret file for all HistoryServer instances that " + + "should accept each other's authentication cookies."); + private static final String HISTORY_SERVER_RETAINED_JOBS_KEY = "historyserver.archive.retained-jobs"; private static final String HISTORY_SERVER_RETAINED_TTL_KEY = @@ -198,5 +280,11 @@ public class HistoryServerOptions { text(CONFIGURE_CONSISTENT)) .build()); + /** Supported HistoryServer web authentication types. */ + public enum HistoryServerWebAuthenticationType { + NONE, + KERBEROS + } + private HistoryServerOptions() {} } diff --git a/flink-core/src/main/java/org/apache/flink/configuration/WebOptions.java b/flink-core/src/main/java/org/apache/flink/configuration/WebOptions.java index e53de835f1baa..cc44dbea40d4d 100644 --- a/flink-core/src/main/java/org/apache/flink/configuration/WebOptions.java +++ b/flink-core/src/main/java/org/apache/flink/configuration/WebOptions.java @@ -123,6 +123,85 @@ public class WebOptions { .withDescription( "Flag indicating whether jobs can be rescaled from the web-frontend."); + /** Authentication type for the JobManager web-frontend. */ + public static final ConfigOption AUTHENTICATION_TYPE = + key("web.authentication.type") + .enumType(WebAuthenticationType.class) + .defaultValue(WebAuthenticationType.NONE) + .withDescription( + "Authentication type for the JobManager web frontend and REST " + + "endpoints."); + + /** Kerberos principal for JobManager web SPNEGO authentication. */ + public static final ConfigOption AUTHENTICATION_KERBEROS_PRINCIPAL = + key("web.authentication.kerberos.principal") + .stringType() + .noDefaultValue() + .withDescription( + "Kerberos principal for JobManager web SPNEGO authentication. " + + "Required when JobManager web authentication type is " + + "KERBEROS. The principal must start with HTTP/. The _HOST " + + "placeholder is replaced with the configured REST address. " + + "Use * to accept all HTTP principals in the keytab."); + + /** Kerberos keytab for JobManager web SPNEGO authentication. */ + public static final ConfigOption AUTHENTICATION_KERBEROS_KEYTAB = + key("web.authentication.kerberos.keytab") + .stringType() + .noDefaultValue() + .withDescription( + "Absolute path to the keytab file for the JobManager web SPNEGO " + + "principal. Required when JobManager web authentication " + + "type is KERBEROS."); + + /** Kerberos auth-to-local rules for JobManager web SPNEGO authentication. */ + public static final ConfigOption AUTHENTICATION_KERBEROS_NAME_RULES = + key("web.authentication.kerberos.name-rules") + .stringType() + .defaultValue("DEFAULT") + .withDescription( + "Kerberos auth-to-local rules used to map authenticated principals to " + + "local user names."); + + /** Validity of JobManager web authentication tokens. */ + public static final ConfigOption AUTHENTICATION_TOKEN_VALIDITY = + key("web.authentication.token.validity") + .durationType() + .defaultValue(Duration.ofHours(10)) + .withDescription( + "Validity period for JobManager web SPNEGO authentication cookies."); + + /** Cookie path for JobManager web authentication tokens. */ + public static final ConfigOption AUTHENTICATION_COOKIE_PATH = + key("web.authentication.cookie.path") + .stringType() + .defaultValue("/") + .withDescription( + "Cookie path for JobManager web SPNEGO authentication cookies."); + + /** Shared signing secret for JobManager web authentication tokens. */ + public static final ConfigOption AUTHENTICATION_SIGNATURE_SECRET = + key("web.authentication.signature.secret") + .stringType() + .noDefaultValue() + .withDescription( + "Shared secret for signing JobManager web SPNEGO authentication " + + "cookies. If neither this option nor the secret-file option " + + "is set, a random process-local secret is generated. " + + "Configure the same secret for all JobManager instances " + + "that should accept each other's authentication cookies."); + + /** File containing the shared signing secret for JobManager web authentication tokens. */ + public static final ConfigOption AUTHENTICATION_SIGNATURE_SECRET_FILE = + key("web.authentication.signature.secret-file") + .stringType() + .noDefaultValue() + .withDescription( + "File containing the shared secret for signing JobManager web SPNEGO " + + "authentication cookies. Configure the same secret file for " + + "all JobManager instances that should accept each other's " + + "authentication cookies."); + /** Config parameter defining the number of checkpoints to remember for recent history. */ public static final ConfigOption CHECKPOINTS_HISTORY_SIZE = key("web.checkpoints.history") @@ -152,4 +231,13 @@ public class WebOptions { /** Not meant to be instantiated. */ private WebOptions() {} + + /** Authentication types supported by the JobManager web-frontend. */ + public enum WebAuthenticationType { + /** Authentication is disabled. */ + NONE, + + /** Kerberos/SPNEGO authentication. */ + KERBEROS + } } diff --git a/flink-dist/pom.xml b/flink-dist/pom.xml index 5be0218dfc889..3217e566ffe1a 100644 --- a/flink-dist/pom.xml +++ b/flink-dist/pom.xml @@ -695,6 +695,9 @@ under the License. org.apache.logging.log4j:* + + org.apache.hadoop:hadoop-auth + org.apache.hadoop.thirdparty:hadoop-shaded-guava diff --git a/flink-dist/src/main/resources/META-INF/NOTICE b/flink-dist/src/main/resources/META-INF/NOTICE index e45b4ccbacdfd..5dca837fb9718 100644 --- a/flink-dist/src/main/resources/META-INF/NOTICE +++ b/flink-dist/src/main/resources/META-INF/NOTICE @@ -7,15 +7,29 @@ The Apache Software Foundation (http://www.apache.org/). This project bundles the following dependencies under the Apache Software License 2.0 (http://www.apache.org/licenses/LICENSE-2.0.txt) - com.google.code.findbugs:jsr305:1.3.9 +- com.nimbusds:nimbus-jose-jwt:10.4 - com.ververica:frocksdbjni:8.10.0-ververica-1.0 - com.ververica:forstjni:0.1.8 +- commons-codec:commons-codec:1.15 - commons-cli:commons-cli:1.5.0 - commons-collections:commons-collections:3.2.2 - commons-io:commons-io:2.15.1 +- commons-logging:commons-logging:1.1.3 - org.apache.commons:commons-compress:1.26.0 - org.apache.commons:commons-lang3:3.18.0 - org.apache.commons:commons-math3:3.6.1 - org.apache.commons:commons-text:1.10.0 +- org.apache.hadoop.thirdparty:hadoop-shaded-guava:1.6.0.1-4.3.0-0 +- org.apache.hadoop:hadoop-auth:3.4.3.1-4.3.0-1 +- org.apache.httpcomponents:httpclient:4.5.13 +- org.apache.httpcomponents:httpcore:4.4.14 +- org.apache.kerby:kerb-core:2.0.3 +- org.apache.kerby:kerb-crypto:2.0.3 +- org.apache.kerby:kerb-util:2.0.3 +- org.apache.kerby:kerby-asn1:2.0.3 +- org.apache.kerby:kerby-config:2.0.3 +- org.apache.kerby:kerby-pkix:2.0.3 +- org.apache.kerby:kerby-util:2.0.3 - org.javassist:javassist:3.24.0-GA - at.yawk.lz4:lz4-java:1.10.3 - org.objenesis:objenesis:3.4 diff --git a/flink-dist/src/main/resources/config.yaml b/flink-dist/src/main/resources/config.yaml index 98ce2fddb93c8..eaba07eb8c1a2 100644 --- a/flink-dist/src/main/resources/config.yaml +++ b/flink-dist/src/main/resources/config.yaml @@ -186,6 +186,24 @@ rest: # bind-port: 8080-8090 # web: +# authentication: +# # Optional Kerberos/SPNEGO authentication for the JobManager Web UI and +# # REST endpoint on port 8081. The default authentication type is NONE. +# # Exposing this endpoint without authentication allows unauthenticated job +# # submission and control operations. +# type: KERBEROS +# kerberos: +# principal: HTTP/_HOST@EXAMPLE.COM +# keytab: /etc/security/keytabs/flink-jobmanager-http.keytab +# # name-rules: DEFAULT +# token: +# validity: 10 h +# cookie: +# path: / +# signature: +# # Configure the same secret file for all JobManager instances that +# # should accept each other's authentication cookies. +# secret-file: /etc/flink/jobmanager-web-auth-secret # submit: # # Flag to specify whether job submission is enabled from the web-based # # runtime monitor. Uncomment to disable. @@ -274,6 +292,27 @@ rest: # # The configuration below must match one of the values set in "security.kerberos.login.contexts" # login-context-name: Client +#============================================================================== +# SQL Gateway REST Security Configuration +#============================================================================== + +# SQL Gateway REST authentication is disabled by default. If the REST endpoint is +# reachable from an untrusted network, enable SPNEGO or protect it with an +# authenticated proxy before allowing clients to create sessions and execute SQL. +# +# sql-gateway: +# endpoint: +# rest: +# address: localhost +# authentication: +# type: KERBEROS +# kerberos: +# principal: HTTP/_HOST@EXAMPLE.COM +# keytab: /etc/security/keytabs/flink-sql-gateway.keytab +# name-rules: DEFAULT +# signature: +# secret-file: /etc/flink/sql-gateway-cookie-secret + #============================================================================== # HistoryServer #============================================================================== @@ -290,13 +329,27 @@ rest: # historyserver: # web: # # The address under which the web-based HistoryServer listens. +# # Binding to 0.0.0.0 exposes archived job metadata to the network. Enable +# # HistoryServer web authentication or protect this endpoint before using +# # a network-facing address. # address: 0.0.0.0 # # The port under which the web-based HistoryServer listens. # port: 8082 +# # Optional Kerberos/SPNEGO authentication for the HistoryServer web UI and +# # REST endpoints. The default authentication type is NONE. +# authentication: +# type: KERBEROS +# kerberos: +# principal: HTTP/_HOST@EXAMPLE.COM +# keytab: /etc/security/keytabs/flink-historyserver.keytab +# # name-rules: DEFAULT +# signature: +# # Configure the same secret file for all HistoryServer instances that +# # should accept each other's authentication cookies. +# secret-file: /etc/flink/historyserver-auth-secret # archive: # fs: # # Comma separated list of directories to monitor for completed jobs. # dir: hdfs:///completed-jobs/ # # Interval in milliseconds for refreshing the monitored directories. # fs.refresh-interval: 10000 - diff --git a/flink-runtime-web/pom.xml b/flink-runtime-web/pom.xml index 10794f27ab3b5..734f7e795c2de 100644 --- a/flink-runtime-web/pom.xml +++ b/flink-runtime-web/pom.xml @@ -80,6 +80,54 @@ under the License. flink-shaded-jackson + + org.apache.hadoop + hadoop-auth + ${flink.hadoop.version} + ${flink.markBundledAsOptional} + + + log4j + log4j + + + org.slf4j + slf4j-log4j12 + + + ch.qos.reload4j + reload4j + + + org.slf4j + slf4j-reload4j + + + io.dropwizard.metrics + metrics-core + + + org.apache.zookeeper + zookeeper + + + org.apache.curator + curator-framework + + + io.netty + netty-handler + + + + + + org.apache.hadoop + hadoop-annotations + ${flink.hadoop.version} + provided + + @@ -108,6 +156,38 @@ under the License. test + + javax.servlet + javax.servlet-api + 3.1.0 + test + + + + org.apache.hadoop + hadoop-minikdc + ${minikdc.version} + test + + + log4j + log4j + + + org.slf4j + slf4j-log4j12 + + + ch.qos.reload4j + reload4j + + + org.slf4j + slf4j-reload4j + + + + @@ -252,6 +332,44 @@ under the License. + + org.apache.maven.plugins + maven-shade-plugin + + + shade-flink + package + + shade + + + + + org.apache.hadoop:hadoop-auth + commons-codec:commons-codec + org.apache.httpcomponents:httpclient + org.apache.httpcomponents:httpcore + org.apache.hadoop.thirdparty:hadoop-shaded-guava + org.apache.kerby:kerb-core + org.apache.kerby:kerby-pkix + org.apache.kerby:kerby-asn1 + org.apache.kerby:kerby-util + org.apache.kerby:kerb-util + org.apache.kerby:kerby-config + org.apache.kerby:kerb-crypto + + + + + org.apache.hadoop + org.apache.flink.runtime.web.shaded.org.apache.hadoop + + + + + + + com.github.eirslett frontend-maven-plugin diff --git a/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/history/HistoryServer.java b/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/history/HistoryServer.java index 51c5a067598b5..3404a75321bd2 100644 --- a/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/history/HistoryServer.java +++ b/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/history/HistoryServer.java @@ -40,6 +40,7 @@ import org.apache.flink.runtime.webmonitor.history.retaining.CompositeJobRetainedStrategy; import org.apache.flink.runtime.webmonitor.utils.LogUrlUtil; import org.apache.flink.runtime.webmonitor.utils.WebFrontendBootstrap; +import org.apache.flink.util.ConfigurationException; import org.apache.flink.util.ExceptionUtils; import org.apache.flink.util.ExecutorUtils; import org.apache.flink.util.FatalExitExceptionHandler; @@ -296,7 +297,7 @@ public void run() { // Life-cycle // ------------------------------------------------------------------------ - void start() throws IOException, InterruptedException { + void start() throws IOException, InterruptedException, ConfigurationException { synchronized (startupShutdownLock) { LOG.info("Starting history server."); @@ -324,6 +325,9 @@ void start() throws IOException, InterruptedException { new GeneratedLogUrlHandler( CompletableFuture.completedFuture(pattern)))); + router.addGet( + HistoryServerAuthenticatedUserHandler.URL, + new HistoryServerAuthenticatedUserHandler()); router.addGet("/:*", new HistoryServerStaticFileServerHandler(webDir)); createDashboardConfigFile(); diff --git a/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/history/HistoryServerAuthenticatedUserHandler.java b/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/history/HistoryServerAuthenticatedUserHandler.java new file mode 100644 index 0000000000000..102c48216a06c --- /dev/null +++ b/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/history/HistoryServerAuthenticatedUserHandler.java @@ -0,0 +1,113 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.runtime.webmonitor.history; + +import org.apache.flink.runtime.rest.handler.router.RoutedRequest; +import org.apache.flink.runtime.rest.handler.util.HandlerUtils; +import org.apache.flink.runtime.rest.messages.ResponseBody; +import org.apache.flink.runtime.webmonitor.history.security.HistoryServerAuthenticatedUser; +import org.apache.flink.runtime.webmonitor.history.security.HistoryServerWebAuthenticationHandler; + +import org.apache.flink.shaded.jackson2.com.fasterxml.jackson.annotation.JsonCreator; +import org.apache.flink.shaded.jackson2.com.fasterxml.jackson.annotation.JsonProperty; +import org.apache.flink.shaded.netty4.io.netty.channel.ChannelHandler; +import org.apache.flink.shaded.netty4.io.netty.channel.ChannelHandlerContext; +import org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpHeaderNames; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpResponseStatus; + +import javax.annotation.Nullable; + +import java.util.Collections; +import java.util.HashMap; +import java.util.Map; + +/** Handler for exposing the current HistoryServer web user to the dashboard. */ +@ChannelHandler.Sharable +public final class HistoryServerAuthenticatedUserHandler + extends SimpleChannelInboundHandler> { + + public static final String URL = "/auth/user"; + + private static final Map NO_STORE_HEADERS = createNoStoreHeaders(); + + @Override + protected void channelRead0(ChannelHandlerContext ctx, RoutedRequest routedRequest) { + HistoryServerAuthenticatedUserResponseBody response = + HistoryServerWebAuthenticationHandler.getAuthenticatedUser(ctx) + .map(HistoryServerAuthenticatedUserResponseBody::authenticated) + .orElseGet(HistoryServerAuthenticatedUserResponseBody::unauthenticated); + + HandlerUtils.sendResponse( + ctx, routedRequest.getRequest(), response, HttpResponseStatus.OK, NO_STORE_HEADERS); + } + + private static Map createNoStoreHeaders() { + Map headers = new HashMap<>(); + headers.put(HttpHeaderNames.CACHE_CONTROL.toString(), "no-store"); + headers.put(HttpHeaderNames.PRAGMA.toString(), "no-cache"); + return Collections.unmodifiableMap(headers); + } + + /** Response body for the current authenticated HistoryServer web user. */ + public static final class HistoryServerAuthenticatedUserResponseBody implements ResponseBody { + + public static final String FIELD_NAME_AUTHENTICATED = "authenticated"; + public static final String FIELD_NAME_USER = "user"; + public static final String FIELD_NAME_PRINCIPAL = "principal"; + public static final String FIELD_NAME_TYPE = "type"; + + @JsonProperty(FIELD_NAME_AUTHENTICATED) + private final boolean authenticated; + + @JsonProperty(FIELD_NAME_USER) + @Nullable + private final String user; + + @JsonProperty(FIELD_NAME_PRINCIPAL) + @Nullable + private final String principal; + + @JsonProperty(FIELD_NAME_TYPE) + @Nullable + private final String type; + + @JsonCreator + public HistoryServerAuthenticatedUserResponseBody( + @JsonProperty(FIELD_NAME_AUTHENTICATED) boolean authenticated, + @Nullable @JsonProperty(FIELD_NAME_USER) String user, + @Nullable @JsonProperty(FIELD_NAME_PRINCIPAL) String principal, + @Nullable @JsonProperty(FIELD_NAME_TYPE) String type) { + this.authenticated = authenticated; + this.user = user; + this.principal = principal; + this.type = type; + } + + static HistoryServerAuthenticatedUserResponseBody authenticated( + HistoryServerAuthenticatedUser user) { + return new HistoryServerAuthenticatedUserResponseBody( + true, user.getUserName(), user.getPrincipal(), user.getType()); + } + + static HistoryServerAuthenticatedUserResponseBody unauthenticated() { + return new HistoryServerAuthenticatedUserResponseBody(false, null, null, null); + } + } +} diff --git a/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/history/security/HistoryServerAuthenticatedUser.java b/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/history/security/HistoryServerAuthenticatedUser.java new file mode 100644 index 0000000000000..d5fdae6ab02ba --- /dev/null +++ b/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/history/security/HistoryServerAuthenticatedUser.java @@ -0,0 +1,47 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.runtime.webmonitor.history.security; + +import org.apache.flink.util.Preconditions; + +/** Authenticated HistoryServer web user for a single HTTP request. */ +public final class HistoryServerAuthenticatedUser { + + private final String userName; + private final String principal; + private final String type; + + HistoryServerAuthenticatedUser(String userName, String principal, String type) { + this.userName = Preconditions.checkNotNull(userName); + this.principal = Preconditions.checkNotNull(principal); + this.type = Preconditions.checkNotNull(type); + } + + public String getUserName() { + return userName; + } + + public String getPrincipal() { + return principal; + } + + public String getType() { + return type; + } +} diff --git a/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/history/security/HistoryServerAuthenticationResult.java b/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/history/security/HistoryServerAuthenticationResult.java new file mode 100644 index 0000000000000..c7250b93dfa25 --- /dev/null +++ b/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/history/security/HistoryServerAuthenticationResult.java @@ -0,0 +1,123 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.runtime.webmonitor.history.security; + +import java.util.Optional; + +/** Result of HistoryServer web authentication for a single HTTP request. */ +final class HistoryServerAuthenticationResult { + + enum Status { + AUTHENTICATED, + UNAUTHORIZED, + FORBIDDEN + } + + private final Status status; + private final Optional userName; + private final Optional principal; + private final Optional type; + private final Optional negotiateToken; + private final Optional signedCookie; + + private HistoryServerAuthenticationResult( + Status status, + Optional userName, + Optional principal, + Optional type, + Optional negotiateToken, + Optional signedCookie) { + this.status = status; + this.userName = userName; + this.principal = principal; + this.type = type; + this.negotiateToken = negotiateToken; + this.signedCookie = signedCookie; + } + + static HistoryServerAuthenticationResult authenticated( + String userName, String principal, String type) { + return new HistoryServerAuthenticationResult( + Status.AUTHENTICATED, + Optional.of(userName), + Optional.of(principal), + Optional.of(type), + Optional.empty(), + Optional.empty()); + } + + static HistoryServerAuthenticationResult authenticated( + String userName, String principal, String type, String signedCookie) { + return new HistoryServerAuthenticationResult( + Status.AUTHENTICATED, + Optional.of(userName), + Optional.of(principal), + Optional.of(type), + Optional.empty(), + Optional.of(signedCookie)); + } + + static HistoryServerAuthenticationResult unauthorized() { + return unauthorized(Optional.empty()); + } + + static HistoryServerAuthenticationResult unauthorized(Optional negotiateToken) { + return new HistoryServerAuthenticationResult( + Status.UNAUTHORIZED, + Optional.empty(), + Optional.empty(), + Optional.empty(), + negotiateToken, + Optional.empty()); + } + + static HistoryServerAuthenticationResult forbidden() { + return new HistoryServerAuthenticationResult( + Status.FORBIDDEN, + Optional.empty(), + Optional.empty(), + Optional.empty(), + Optional.empty(), + Optional.empty()); + } + + Status getStatus() { + return status; + } + + Optional getUserName() { + return userName; + } + + Optional getPrincipal() { + return principal; + } + + Optional getType() { + return type; + } + + Optional getNegotiateToken() { + return negotiateToken; + } + + Optional getSignedCookie() { + return signedCookie; + } +} diff --git a/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/history/security/HistoryServerAuthenticationTokenSigner.java b/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/history/security/HistoryServerAuthenticationTokenSigner.java new file mode 100644 index 0000000000000..11bdd649e90ca --- /dev/null +++ b/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/history/security/HistoryServerAuthenticationTokenSigner.java @@ -0,0 +1,106 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.runtime.webmonitor.history.security; + +import org.apache.hadoop.security.authentication.client.AuthenticationException; +import org.apache.hadoop.security.authentication.server.AuthenticationToken; + +import javax.crypto.Mac; +import javax.crypto.spec.SecretKeySpec; + +import java.nio.charset.StandardCharsets; +import java.security.InvalidKeyException; +import java.security.MessageDigest; +import java.security.NoSuchAlgorithmException; +import java.util.Base64; +import java.util.Objects; + +/** Signs and verifies Hadoop Auth compatible HistoryServer authentication tokens. */ +final class HistoryServerAuthenticationTokenSigner { + + private static final String SIGNATURE_SEPARATOR = "&s="; + private static final String SIGNING_ALGORITHM = "HmacSHA256"; + + private final byte[] secret; + + HistoryServerAuthenticationTokenSigner(byte[] secret) { + this.secret = Objects.requireNonNull(secret, "secret must not be null").clone(); + if (this.secret.length == 0) { + throw new IllegalArgumentException("secret must not be empty"); + } + } + + String sign(AuthenticationToken token) { + return sign(token.toString()); + } + + String sign(String token) { + if (token == null || token.isEmpty()) { + throw new IllegalArgumentException("token must not be empty"); + } + return token + SIGNATURE_SEPARATOR + computeSignature(token); + } + + AuthenticationToken verifyAndExtract(String signedToken) throws AuthenticationException { + String token = verifyAndExtractRaw(stripQuotes(signedToken)); + return AuthenticationToken.parse(token); + } + + String verifyAndExtractRaw(String signedToken) throws AuthenticationException { + if (signedToken == null || signedToken.isEmpty()) { + throw new AuthenticationException("Missing authentication token"); + } + int index = signedToken.lastIndexOf(SIGNATURE_SEPARATOR); + if (index == -1) { + throw new AuthenticationException("Invalid signed authentication token"); + } + + String token = signedToken.substring(0, index); + String expectedSignature = computeSignature(token); + String actualSignature = signedToken.substring(index + SIGNATURE_SEPARATOR.length()); + + if (!MessageDigest.isEqual( + expectedSignature.getBytes(StandardCharsets.UTF_8), + actualSignature.getBytes(StandardCharsets.UTF_8))) { + throw new AuthenticationException("Invalid authentication token signature"); + } + return token; + } + + private String computeSignature(String token) { + try { + Mac mac = Mac.getInstance(SIGNING_ALGORITHM); + mac.init(new SecretKeySpec(secret, SIGNING_ALGORITHM)); + byte[] signature = mac.doFinal(token.getBytes(StandardCharsets.UTF_8)); + return Base64.getEncoder().encodeToString(signature); + } catch (NoSuchAlgorithmException | InvalidKeyException e) { + throw new IllegalStateException("Could not sign authentication token.", e); + } + } + + private static String stripQuotes(String value) { + if (value != null + && value.length() >= 2 + && value.startsWith("\"") + && value.endsWith("\"")) { + return value.substring(1, value.length() - 1); + } + return value; + } +} diff --git a/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/history/security/HistoryServerSpnegoAuthenticator.java b/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/history/security/HistoryServerSpnegoAuthenticator.java new file mode 100644 index 0000000000000..f8e4fedba5770 --- /dev/null +++ b/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/history/security/HistoryServerSpnegoAuthenticator.java @@ -0,0 +1,377 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.runtime.webmonitor.history.security; + +import org.apache.flink.configuration.Configuration; +import org.apache.flink.runtime.security.KerberosUtils; +import org.apache.flink.util.ConfigurationException; + +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpHeaderNames; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpRequest; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.cookie.Cookie; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.cookie.ServerCookieDecoder; + +import org.apache.hadoop.security.authentication.client.AuthenticatedURL; +import org.apache.hadoop.security.authentication.client.AuthenticationException; +import org.apache.hadoop.security.authentication.client.KerberosAuthenticator; +import org.apache.hadoop.security.authentication.server.AuthenticationToken; +import org.apache.hadoop.security.authentication.util.KerberosName; +import org.apache.hadoop.security.authentication.util.KerberosUtil; +import org.ietf.jgss.GSSContext; +import org.ietf.jgss.GSSCredential; +import org.ietf.jgss.GSSException; +import org.ietf.jgss.GSSManager; +import org.ietf.jgss.Oid; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import javax.security.auth.Subject; +import javax.security.auth.callback.CallbackHandler; +import javax.security.auth.login.AppConfigurationEntry; +import javax.security.auth.login.LoginContext; +import javax.security.auth.login.LoginException; + +import java.io.File; +import java.io.IOException; +import java.security.PrivilegedActionException; +import java.security.PrivilegedExceptionAction; +import java.time.Clock; +import java.util.Base64; +import java.util.Collections; +import java.util.HashMap; +import java.util.Map; +import java.util.Optional; +import java.util.Set; +import java.util.regex.Pattern; + +/** Performs Kerberos/SPNEGO authentication for the Flink HistoryServer web endpoint. */ +final class HistoryServerSpnegoAuthenticator { + + static final String TOKEN_TYPE = "kerberos"; + + private static final Logger LOG = + LoggerFactory.getLogger(HistoryServerSpnegoAuthenticator.class); + private static final Pattern HTTP_PRINCIPAL_PATTERN = Pattern.compile("HTTP/.*"); + + private final Map serverSubjects; + private final GSSManager gssManager; + private final HistoryServerAuthenticationTokenSigner tokenSigner; + private final long tokenValidityMillis; + private final String cookiePath; + private final Clock clock; + + private HistoryServerSpnegoAuthenticator( + Map serverSubjects, + GSSManager gssManager, + HistoryServerAuthenticationTokenSigner tokenSigner, + long tokenValidityMillis, + String cookiePath, + Clock clock) { + this.serverSubjects = serverSubjects; + this.gssManager = gssManager; + this.tokenSigner = tokenSigner; + this.tokenValidityMillis = tokenValidityMillis; + this.cookiePath = cookiePath; + this.clock = clock; + } + + static Optional fromConfiguration(Configuration configuration) + throws ConfigurationException { + Optional config = + HistoryServerWebAuthenticationConfig.from(configuration); + if (!config.isPresent()) { + return Optional.empty(); + } + return Optional.of(fromConfig(config.get(), Clock.systemUTC())); + } + + static HistoryServerSpnegoAuthenticator fromConfig( + HistoryServerWebAuthenticationConfig config, Clock clock) + throws ConfigurationException { + Map serverSubjects = createServerSubjects(config); + return new HistoryServerSpnegoAuthenticator( + serverSubjects, + GSSManager.getInstance(), + new HistoryServerAuthenticationTokenSigner(config.getSignatureSecret()), + config.getTokenValidity().toMillis(), + config.getCookiePath(), + clock); + } + + HistoryServerAuthenticationResult authenticate(HttpRequest request) { + Optional cookieValue = getAuthenticationCookie(request); + if (cookieValue.isPresent()) { + try { + AuthenticationToken token = tokenSigner.verifyAndExtract(cookieValue.get()); + if (TOKEN_TYPE.equals(token.getType()) && !token.isExpired()) { + return HistoryServerAuthenticationResult.authenticated( + token.getUserName(), token.getName(), token.getType()); + } + LOG.debug("Ignoring invalid or expired HistoryServer authentication cookie."); + } catch (AuthenticationException | IllegalArgumentException e) { + LOG.debug("Ignoring invalid HistoryServer authentication cookie.", e); + } + } + + String authorization = request.headers().get(KerberosAuthenticator.AUTHORIZATION); + if (authorization == null || !startsWithNegotiate(authorization)) { + return HistoryServerAuthenticationResult.unauthorized(); + } + + String encodedToken = + authorization.substring(KerberosAuthenticator.NEGOTIATE.length()).trim(); + if (encodedToken.isEmpty()) { + return HistoryServerAuthenticationResult.forbidden(); + } + + final byte[] clientToken; + try { + clientToken = Base64.getDecoder().decode(encodedToken); + } catch (IllegalArgumentException e) { + LOG.debug("Received malformed SPNEGO token.", e); + return HistoryServerAuthenticationResult.forbidden(); + } + + try { + String serverPrincipal = KerberosUtil.getTokenServerName(clientToken); + if (!serverPrincipal.startsWith("HTTP/")) { + LOG.warn("Rejecting SPNEGO token for non-HTTP service principal."); + return HistoryServerAuthenticationResult.forbidden(); + } + Subject serverSubject = serverSubjects.get(serverPrincipal); + if (serverSubject == null) { + LOG.warn( + "Rejecting SPNEGO token for an unconfigured HistoryServer service principal."); + return HistoryServerAuthenticationResult.forbidden(); + } + return Subject.doAs( + serverSubject, + (PrivilegedExceptionAction) + () -> authenticate(serverPrincipal, clientToken)); + } catch (PrivilegedActionException e) { + LOG.debug("SPNEGO authentication failed.", e.getException()); + return HistoryServerAuthenticationResult.forbidden(); + } catch (RuntimeException e) { + LOG.debug("SPNEGO authentication failed.", e); + return HistoryServerAuthenticationResult.forbidden(); + } + } + + String createAuthenticationCookie(String signedToken, boolean secure) { + StringBuilder cookie = + new StringBuilder(AuthenticatedURL.AUTH_COOKIE) + .append("=\"") + .append(signedToken) + .append("\"; Path=") + .append(cookiePath) + .append("; HttpOnly"); + if (secure) { + cookie.append("; Secure"); + } + return cookie.toString(); + } + + String createExpiredAuthenticationCookie(boolean secure) { + StringBuilder cookie = + new StringBuilder(AuthenticatedURL.AUTH_COOKIE) + .append("=; Path=") + .append(cookiePath) + .append("; Max-Age=0; HttpOnly"); + if (secure) { + cookie.append("; Secure"); + } + return cookie.toString(); + } + + HistoryServerAuthenticationTokenSigner getTokenSigner() { + return tokenSigner; + } + + private HistoryServerAuthenticationResult authenticate( + String serverPrincipal, byte[] clientToken) throws GSSException, IOException { + GSSContext gssContext = null; + GSSCredential gssCredential = null; + try { + gssCredential = + gssManager.createCredential( + gssManager.createName( + serverPrincipal, KerberosUtil.NT_GSS_KRB5_PRINCIPAL_OID), + GSSCredential.INDEFINITE_LIFETIME, + new Oid[] { + KerberosUtil.GSS_SPNEGO_MECH_OID, KerberosUtil.GSS_KRB5_MECH_OID + }, + GSSCredential.ACCEPT_ONLY); + gssContext = gssManager.createContext(gssCredential); + byte[] serverToken = gssContext.acceptSecContext(clientToken, 0, clientToken.length); + if (!gssContext.isEstablished()) { + return HistoryServerAuthenticationResult.unauthorized(encode(serverToken)); + } + + String clientPrincipal = gssContext.getSrcName().toString(); + String userName = new KerberosName(clientPrincipal).getShortName(); + AuthenticationToken authenticationToken = + new AuthenticationToken(userName, clientPrincipal, TOKEN_TYPE); + authenticationToken.setExpires(clock.millis() + tokenValidityMillis); + return HistoryServerAuthenticationResult.authenticated( + userName, + clientPrincipal, + authenticationToken.getType(), + tokenSigner.sign(authenticationToken)); + } finally { + if (gssContext != null) { + gssContext.dispose(); + } + if (gssCredential != null) { + gssCredential.dispose(); + } + } + } + + private static Optional encode(byte[] token) { + if (token == null || token.length == 0) { + return Optional.empty(); + } + return Optional.of(Base64.getEncoder().encodeToString(token)); + } + + private static boolean startsWithNegotiate(String authorization) { + return authorization.regionMatches( + true, + 0, + KerberosAuthenticator.NEGOTIATE, + 0, + KerberosAuthenticator.NEGOTIATE.length()); + } + + private static Optional getAuthenticationCookie(HttpRequest request) { + for (String cookieHeader : request.headers().getAll(HttpHeaderNames.COOKIE)) { + try { + Set cookies = ServerCookieDecoder.STRICT.decode(cookieHeader); + for (Cookie cookie : cookies) { + if (AuthenticatedURL.AUTH_COOKIE.equals(cookie.name())) { + return Optional.of(cookie.value()); + } + } + } catch (IllegalArgumentException e) { + LOG.debug("Ignoring malformed HistoryServer Cookie header.", e); + } + } + return Optional.empty(); + } + + private static Map createServerSubjects( + HistoryServerWebAuthenticationConfig config) throws ConfigurationException { + Map subjects = new HashMap<>(); + for (String principal : resolveServerPrincipals(config)) { + subjects.put( + principal, loginServerSubject(config.getKerberosKeytab().toFile(), principal)); + } + configureKerberosNameRules(config); + return Collections.unmodifiableMap(subjects); + } + + private static void configureKerberosNameRules(HistoryServerWebAuthenticationConfig config) + throws ConfigurationException { + try { + KerberosName.setRules(config.getKerberosNameRules().orElse("DEFAULT")); + KerberosName.setRuleMechanism(KerberosName.DEFAULT_MECHANISM); + } catch (IllegalArgumentException e) { + throw new ConfigurationException( + "Invalid HistoryServer SPNEGO Kerberos name rules.", e); + } + } + + private static Subject loginServerSubject(File keytabFile, String principal) + throws ConfigurationException { + Subject subject = new Subject(); + LoginContext loginContext; + try { + loginContext = + new LoginContext( + "", + subject, + (CallbackHandler) null, + new javax.security.auth.login.Configuration() { + @Override + public AppConfigurationEntry[] getAppConfigurationEntry( + String name) { + return new AppConfigurationEntry[] { + acceptorKeytabEntry(keytabFile, principal) + }; + } + }); + loginContext.login(); + } catch (LoginException e) { + throw new ConfigurationException( + "Could not log in HistoryServer SPNEGO principal " + + principal + + " from keytab " + + keytabFile, + e); + } + LOG.info("Using HistoryServer SPNEGO principal {} from keytab {}", principal, keytabFile); + return loginContext.getSubject(); + } + + private static AppConfigurationEntry acceptorKeytabEntry(File keytabFile, String principal) { + Map kerberosOptions = new HashMap<>(); + if (KerberosUtils.getKrb5LoginModuleName().contains("ibm")) { + kerberosOptions.put("useKeytab", keytabFile.toURI().toString()); + kerberosOptions.put("credsType", "both"); + } else { + kerberosOptions.put("keyTab", keytabFile.getAbsolutePath()); + kerberosOptions.put("doNotPrompt", "true"); + kerberosOptions.put("useKeyTab", "true"); + kerberosOptions.put("storeKey", "true"); + kerberosOptions.put("isInitiator", "false"); + } + + kerberosOptions.put("principal", principal); + kerberosOptions.put("refreshKrb5Config", "true"); + + return new AppConfigurationEntry( + KerberosUtils.getKrb5LoginModuleName(), + AppConfigurationEntry.LoginModuleControlFlag.REQUIRED, + kerberosOptions); + } + + private static String[] resolveServerPrincipals(HistoryServerWebAuthenticationConfig config) + throws ConfigurationException { + if (!"*".equals(config.getKerberosPrincipal())) { + return new String[] {config.getKerberosPrincipal()}; + } + try { + String[] principals = + KerberosUtil.getPrincipalNames( + config.getKerberosKeytab().toString(), HTTP_PRINCIPAL_PATTERN); + if (principals.length == 0) { + throw new ConfigurationException( + "No HTTP principals were found in " + + config.getKerberosKeytab() + + " for HistoryServer SPNEGO authentication."); + } + return principals; + } catch (IOException e) { + throw new ConfigurationException( + "Could not read HistoryServer SPNEGO principals from " + + config.getKerberosKeytab(), + e); + } + } +} diff --git a/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/history/security/HistoryServerWebAuthenticationConfig.java b/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/history/security/HistoryServerWebAuthenticationConfig.java new file mode 100644 index 0000000000000..87119cbbafd9b --- /dev/null +++ b/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/history/security/HistoryServerWebAuthenticationConfig.java @@ -0,0 +1,269 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.runtime.webmonitor.history.security; + +import org.apache.flink.configuration.Configuration; +import org.apache.flink.configuration.HistoryServerOptions; +import org.apache.flink.configuration.HistoryServerOptions.HistoryServerWebAuthenticationType; +import org.apache.flink.util.ConfigurationException; + +import java.io.IOException; +import java.net.InetAddress; +import java.net.UnknownHostException; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.Paths; +import java.security.SecureRandom; +import java.time.Duration; +import java.util.Locale; +import java.util.Optional; + +import static org.apache.flink.configuration.HistoryServerOptions.HISTORY_SERVER_WEB_AUTHENTICATION_COOKIE_PATH; +import static org.apache.flink.configuration.HistoryServerOptions.HISTORY_SERVER_WEB_AUTHENTICATION_KERBEROS_KEYTAB; +import static org.apache.flink.configuration.HistoryServerOptions.HISTORY_SERVER_WEB_AUTHENTICATION_KERBEROS_NAME_RULES; +import static org.apache.flink.configuration.HistoryServerOptions.HISTORY_SERVER_WEB_AUTHENTICATION_KERBEROS_PRINCIPAL; +import static org.apache.flink.configuration.HistoryServerOptions.HISTORY_SERVER_WEB_AUTHENTICATION_SIGNATURE_SECRET; +import static org.apache.flink.configuration.HistoryServerOptions.HISTORY_SERVER_WEB_AUTHENTICATION_SIGNATURE_SECRET_FILE; +import static org.apache.flink.configuration.HistoryServerOptions.HISTORY_SERVER_WEB_AUTHENTICATION_TOKEN_VALIDITY; +import static org.apache.flink.configuration.HistoryServerOptions.HISTORY_SERVER_WEB_AUTHENTICATION_TYPE; + +/** Parsed and validated HistoryServer web authentication configuration. */ +final class HistoryServerWebAuthenticationConfig { + + private static final int RANDOM_SECRET_BYTES = 32; + + private final String kerberosPrincipal; + private final Path kerberosKeytab; + private final Optional kerberosNameRules; + private final Duration tokenValidity; + private final String cookiePath; + private final byte[] signatureSecret; + + private HistoryServerWebAuthenticationConfig( + String kerberosPrincipal, + Path kerberosKeytab, + Optional kerberosNameRules, + Duration tokenValidity, + String cookiePath, + byte[] signatureSecret) { + this.kerberosPrincipal = kerberosPrincipal; + this.kerberosKeytab = kerberosKeytab; + this.kerberosNameRules = kerberosNameRules; + this.tokenValidity = tokenValidity; + this.cookiePath = cookiePath; + this.signatureSecret = signatureSecret.clone(); + } + + static Optional from(Configuration configuration) + throws ConfigurationException { + HistoryServerWebAuthenticationType type = getAuthenticationType(configuration); + if (type == HistoryServerWebAuthenticationType.NONE) { + return Optional.empty(); + } + if (type != HistoryServerWebAuthenticationType.KERBEROS) { + throw new ConfigurationException( + "Unsupported " + + HISTORY_SERVER_WEB_AUTHENTICATION_TYPE.key() + + " value: " + + type); + } + + return Optional.of( + new HistoryServerWebAuthenticationConfig( + resolvePrincipal( + required( + configuration, + HISTORY_SERVER_WEB_AUTHENTICATION_KERBEROS_PRINCIPAL)), + resolveKeytab( + required( + configuration, + HISTORY_SERVER_WEB_AUTHENTICATION_KERBEROS_KEYTAB)), + optionalNonBlank( + configuration, + HISTORY_SERVER_WEB_AUTHENTICATION_KERBEROS_NAME_RULES), + validateTokenValidity( + configuration.get( + HISTORY_SERVER_WEB_AUTHENTICATION_TOKEN_VALIDITY)), + validateCookiePath( + configuration.get(HISTORY_SERVER_WEB_AUTHENTICATION_COOKIE_PATH)), + resolveSignatureSecret(configuration))); + } + + String getKerberosPrincipal() { + return kerberosPrincipal; + } + + Path getKerberosKeytab() { + return kerberosKeytab; + } + + Optional getKerberosNameRules() { + return kerberosNameRules; + } + + Duration getTokenValidity() { + return tokenValidity; + } + + String getCookiePath() { + return cookiePath; + } + + byte[] getSignatureSecret() { + return signatureSecret.clone(); + } + + private static HistoryServerWebAuthenticationType getAuthenticationType( + Configuration configuration) throws ConfigurationException { + try { + return configuration.get(HISTORY_SERVER_WEB_AUTHENTICATION_TYPE); + } catch (IllegalArgumentException e) { + throw new ConfigurationException( + "Unsupported " + HISTORY_SERVER_WEB_AUTHENTICATION_TYPE.key() + " value.", e); + } + } + + private static String resolvePrincipal(String principal) throws ConfigurationException { + if ("*".equals(principal)) { + return principal; + } + String resolvedPrincipal = principal.replace("_HOST", getLocalHostName()); + if (!resolvedPrincipal.startsWith("HTTP/")) { + throw new ConfigurationException( + HistoryServerOptions.HISTORY_SERVER_WEB_AUTHENTICATION_KERBEROS_PRINCIPAL.key() + + " must start with HTTP/ or be '*'."); + } + return resolvedPrincipal; + } + + private static String getLocalHostName() throws ConfigurationException { + try { + return InetAddress.getLocalHost().getCanonicalHostName().toLowerCase(Locale.ROOT); + } catch (UnknownHostException e) { + throw new ConfigurationException( + "Could not resolve local hostname for HistoryServer SPNEGO principal.", e); + } + } + + private static Path resolveKeytab(String keytab) throws ConfigurationException { + Path keytabPath = Paths.get(keytab); + if (!Files.isRegularFile(keytabPath)) { + throw new ConfigurationException( + HISTORY_SERVER_WEB_AUTHENTICATION_KERBEROS_KEYTAB.key() + + " does not point to a regular file: " + + keytabPath); + } + return keytabPath; + } + + private static Duration validateTokenValidity(Duration tokenValidity) + throws ConfigurationException { + if (tokenValidity == null || tokenValidity.isZero() || tokenValidity.isNegative()) { + throw new ConfigurationException( + HISTORY_SERVER_WEB_AUTHENTICATION_TOKEN_VALIDITY.key() + + " must be greater than 0."); + } + return tokenValidity; + } + + private static String validateCookiePath(String cookiePath) throws ConfigurationException { + if (cookiePath == null || cookiePath.trim().isEmpty() || !cookiePath.startsWith("/")) { + throw new ConfigurationException( + HISTORY_SERVER_WEB_AUTHENTICATION_COOKIE_PATH.key() + + " must be a non-empty absolute path."); + } + return cookiePath; + } + + private static byte[] resolveSignatureSecret(Configuration configuration) + throws ConfigurationException { + Optional configuredSecret = + optionalNonBlank(configuration, HISTORY_SERVER_WEB_AUTHENTICATION_SIGNATURE_SECRET); + Optional configuredSecretFile = + optionalNonBlank( + configuration, HISTORY_SERVER_WEB_AUTHENTICATION_SIGNATURE_SECRET_FILE); + + if (configuredSecret.isPresent() && configuredSecretFile.isPresent()) { + throw new ConfigurationException( + "Only one of " + + HISTORY_SERVER_WEB_AUTHENTICATION_SIGNATURE_SECRET.key() + + " and " + + HISTORY_SERVER_WEB_AUTHENTICATION_SIGNATURE_SECRET_FILE.key() + + " may be configured."); + } + + if (configuredSecret.isPresent()) { + return configuredSecret.get().getBytes(StandardCharsets.UTF_8); + } + if (configuredSecretFile.isPresent()) { + return readSecretFile(configuredSecretFile.get()); + } + return createRandomSecret(); + } + + private static byte[] readSecretFile(String secretFile) throws ConfigurationException { + try { + String secret = + new String(Files.readAllBytes(Paths.get(secretFile)), StandardCharsets.UTF_8) + .trim(); + if (secret.isEmpty()) { + throw new ConfigurationException( + HISTORY_SERVER_WEB_AUTHENTICATION_SIGNATURE_SECRET_FILE.key() + + " must not be empty."); + } + return secret.getBytes(StandardCharsets.UTF_8); + } catch (IOException e) { + throw new ConfigurationException( + "Could not read " + + HISTORY_SERVER_WEB_AUTHENTICATION_SIGNATURE_SECRET_FILE.key() + + ": " + + secretFile, + e); + } + } + + private static byte[] createRandomSecret() { + byte[] secret = new byte[RANDOM_SECRET_BYTES]; + new SecureRandom().nextBytes(secret); + return secret; + } + + private static String required( + Configuration configuration, org.apache.flink.configuration.ConfigOption option) + throws ConfigurationException { + return optionalNonBlank(configuration, option) + .orElseThrow( + () -> + new ConfigurationException( + option.key() + + " must be configured when " + + HISTORY_SERVER_WEB_AUTHENTICATION_TYPE.key() + + " is KERBEROS.")); + } + + private static Optional optionalNonBlank( + Configuration configuration, + org.apache.flink.configuration.ConfigOption option) { + return configuration + .getOptional(option) + .map(String::trim) + .filter(value -> !value.isEmpty()); + } +} diff --git a/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/history/security/HistoryServerWebAuthenticationHandler.java b/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/history/security/HistoryServerWebAuthenticationHandler.java new file mode 100644 index 0000000000000..0161d8e7a824c --- /dev/null +++ b/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/history/security/HistoryServerWebAuthenticationHandler.java @@ -0,0 +1,229 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.runtime.webmonitor.history.security; + +import org.apache.flink.configuration.Configuration; +import org.apache.flink.util.ConfigurationException; + +import org.apache.flink.shaded.netty4.io.netty.buffer.Unpooled; +import org.apache.flink.shaded.netty4.io.netty.channel.ChannelDuplexHandler; +import org.apache.flink.shaded.netty4.io.netty.channel.ChannelFutureListener; +import org.apache.flink.shaded.netty4.io.netty.channel.ChannelHandler; +import org.apache.flink.shaded.netty4.io.netty.channel.ChannelHandlerContext; +import org.apache.flink.shaded.netty4.io.netty.channel.ChannelPromise; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.DefaultFullHttpResponse; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.FullHttpResponse; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpHeaderNames; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpHeaderValues; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpRequest; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpResponse; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpResponseStatus; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpUtil; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpVersion; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.LastHttpContent; +import org.apache.flink.shaded.netty4.io.netty.util.AttributeKey; +import org.apache.flink.shaded.netty4.io.netty.util.ReferenceCountUtil; + +import org.apache.hadoop.security.authentication.client.KerberosAuthenticator; + +import java.util.ArrayDeque; +import java.util.Optional; +import java.util.Queue; + +/** Netty handler that enforces HistoryServer web authentication when configured. */ +public final class HistoryServerWebAuthenticationHandler extends ChannelDuplexHandler { + + private static final AttributeKey> PENDING_AUTH_COOKIES = + AttributeKey.valueOf("history-server-pending-auth-cookies"); + private static final AttributeKey AUTHENTICATED_USER = + AttributeKey.valueOf("history-server-authenticated-user"); + + private final HistoryServerSpnegoAuthenticator authenticator; + private final boolean secureCookie; + + private HistoryServerWebAuthenticationHandler( + HistoryServerSpnegoAuthenticator authenticator, boolean secureCookie) { + this.authenticator = authenticator; + this.secureCookie = secureCookie; + } + + public static Optional createFactory(Configuration configuration, boolean secureCookie) + throws ConfigurationException { + return HistoryServerSpnegoAuthenticator.fromConfiguration(configuration) + .map( + authenticator -> + () -> + new HistoryServerWebAuthenticationHandler( + authenticator, secureCookie)); + } + + @Override + public void channelRead(ChannelHandlerContext ctx, Object msg) { + if (!(msg instanceof HttpRequest)) { + if (msg instanceof LastHttpContent) { + try { + ctx.fireChannelRead(msg); + } finally { + clearAuthenticatedUser(ctx); + } + } else { + ctx.fireChannelRead(msg); + } + return; + } + + HttpRequest request = (HttpRequest) msg; + HistoryServerAuthenticationResult authenticationResult = + authenticator.authenticate(request); + + switch (authenticationResult.getStatus()) { + case AUTHENTICATED: + authenticationResult + .getSignedCookie() + .map( + cookie -> + authenticator.createAuthenticationCookie( + cookie, secureCookie)) + .ifPresent(cookie -> enqueuePendingCookie(ctx, cookie)); + setAuthenticatedUser(ctx, authenticationResult); + try { + ctx.fireChannelRead(msg); + } finally { + if (msg instanceof LastHttpContent) { + clearAuthenticatedUser(ctx); + } + } + break; + case UNAUTHORIZED: + clearAuthenticatedUser(ctx); + writeResponse( + ctx, + request, + createUnauthorizedResponse(authenticationResult.getNegotiateToken())); + break; + case FORBIDDEN: + clearAuthenticatedUser(ctx); + writeResponse(ctx, request, createForbiddenResponse()); + break; + default: + throw new IllegalStateException( + "Unknown authentication status: " + authenticationResult.getStatus()); + } + } + + @Override + public void write(ChannelHandlerContext ctx, Object msg, ChannelPromise promise) + throws Exception { + if (msg instanceof HttpResponse) { + Queue pendingCookies = ctx.channel().attr(PENDING_AUTH_COOKIES).get(); + if (pendingCookies != null) { + String cookie = pendingCookies.poll(); + if (cookie != null) { + ((HttpResponse) msg).headers().add(HttpHeaderNames.SET_COOKIE, cookie); + } + } + } + super.write(ctx, msg, promise); + } + + private void enqueuePendingCookie(ChannelHandlerContext ctx, String cookie) { + Queue pendingCookies = ctx.channel().attr(PENDING_AUTH_COOKIES).get(); + if (pendingCookies == null) { + pendingCookies = new ArrayDeque<>(); + ctx.channel().attr(PENDING_AUTH_COOKIES).set(pendingCookies); + } + pendingCookies.add(cookie); + } + + private void setAuthenticatedUser( + ChannelHandlerContext ctx, HistoryServerAuthenticationResult authenticationResult) { + ctx.channel() + .attr(AUTHENTICATED_USER) + .set( + new HistoryServerAuthenticatedUser( + authenticationResult + .getUserName() + .orElseThrow(IllegalStateException::new), + authenticationResult + .getPrincipal() + .orElseThrow(IllegalStateException::new), + authenticationResult + .getType() + .orElseThrow(IllegalStateException::new))); + } + + private static void clearAuthenticatedUser(ChannelHandlerContext ctx) { + ctx.channel().attr(AUTHENTICATED_USER).set(null); + } + + public static Optional getAuthenticatedUser( + ChannelHandlerContext ctx) { + return Optional.ofNullable(ctx.channel().attr(AUTHENTICATED_USER).get()); + } + + private FullHttpResponse createUnauthorizedResponse(Optional negotiateToken) { + FullHttpResponse response = + new DefaultFullHttpResponse( + HttpVersion.HTTP_1_1, + HttpResponseStatus.UNAUTHORIZED, + Unpooled.EMPTY_BUFFER); + response.headers().set(HttpHeaderNames.CONTENT_LENGTH, 0); + response.headers().set(HttpHeaderNames.WWW_AUTHENTICATE, negotiateValue(negotiateToken)); + response.headers() + .add( + HttpHeaderNames.SET_COOKIE, + authenticator.createExpiredAuthenticationCookie(secureCookie)); + return response; + } + + private FullHttpResponse createForbiddenResponse() { + FullHttpResponse response = + new DefaultFullHttpResponse( + HttpVersion.HTTP_1_1, HttpResponseStatus.FORBIDDEN, Unpooled.EMPTY_BUFFER); + response.headers().set(HttpHeaderNames.CONTENT_LENGTH, 0); + response.headers() + .add( + HttpHeaderNames.SET_COOKIE, + authenticator.createExpiredAuthenticationCookie(secureCookie)); + return response; + } + + private static String negotiateValue(Optional negotiateToken) { + return negotiateToken + .map(token -> KerberosAuthenticator.NEGOTIATE + " " + token) + .orElse(KerberosAuthenticator.NEGOTIATE); + } + + private static void writeResponse( + ChannelHandlerContext ctx, HttpRequest request, FullHttpResponse response) { + boolean keepAlive = HttpUtil.isKeepAlive(request); + ReferenceCountUtil.release(request); + if (keepAlive) { + response.headers().set(HttpHeaderNames.CONNECTION, HttpHeaderValues.KEEP_ALIVE); + ctx.writeAndFlush(response); + } else { + ctx.writeAndFlush(response).addListener(ChannelFutureListener.CLOSE); + } + } + + /** Factory for per-channel HistoryServer authentication handlers. */ + public interface Factory { + ChannelHandler createHandler(); + } +} diff --git a/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/utils/WebFrontendBootstrap.java b/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/utils/WebFrontendBootstrap.java index 78095b8453cc5..037aa51b97dd1 100644 --- a/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/utils/WebFrontendBootstrap.java +++ b/flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/utils/WebFrontendBootstrap.java @@ -28,6 +28,7 @@ import org.apache.flink.runtime.rest.handler.router.RouterHandler; import org.apache.flink.runtime.webmonitor.HttpRequestHandler; import org.apache.flink.runtime.webmonitor.PipelineErrorHandler; +import org.apache.flink.runtime.webmonitor.history.security.HistoryServerWebAuthenticationHandler; import org.apache.flink.util.ConfigurationException; import org.apache.flink.util.Preconditions; @@ -71,6 +72,8 @@ public class WebFrontendBootstrap { private final String restAddress; private final int maxContentLength; private final Map responseHeaders; + private final Optional + historyServerWebAuthenticationHandlerFactory; @VisibleForTesting List inboundChannelHandlerFactories; public WebFrontendBootstrap( @@ -81,13 +84,16 @@ public WebFrontendBootstrap( String configuredAddress, int configuredPort, final Configuration config) - throws InterruptedException, UnknownHostException { + throws InterruptedException, UnknownHostException, ConfigurationException { this.router = Preconditions.checkNotNull(router); this.log = Preconditions.checkNotNull(log); this.uploadDir = directory; this.maxContentLength = config.get(SERVER_MAX_CONTENT_LENGTH); this.responseHeaders = new HashMap<>(); + this.historyServerWebAuthenticationHandlerFactory = + HistoryServerWebAuthenticationHandler.createFactory( + config, serverSSLFactory != null); inboundChannelHandlerFactories = new ArrayList<>(); ServiceLoader loader = ServiceLoader.load(InboundChannelHandlerFactory.class); @@ -124,8 +130,12 @@ protected void initChannel(SocketChannel ch) throws ConfigurationException { serverSSLFactory.createNettySSLHandler(ch.alloc())); } + ch.pipeline().addLast(new HttpServerCodec()); + + historyServerWebAuthenticationHandlerFactory.ifPresent( + factory -> ch.pipeline().addLast(factory.createHandler())); + ch.pipeline() - .addLast(new HttpServerCodec()) .addLast(new HttpRequestHandler(uploadDir)) .addLast( new FlinkHttpObjectAggregator( diff --git a/flink-runtime-web/src/main/resources/META-INF/NOTICE b/flink-runtime-web/src/main/resources/META-INF/NOTICE index 1951c03335636..f415df2a67853 100644 --- a/flink-runtime-web/src/main/resources/META-INF/NOTICE +++ b/flink-runtime-web/src/main/resources/META-INF/NOTICE @@ -4,6 +4,21 @@ Copyright 2014-2025 The Apache Software Foundation This product includes software developed at The Apache Software Foundation (http://www.apache.org/). +This project bundles the following dependencies under the Apache Software License 2.0 (http://www.apache.org/licenses/LICENSE-2.0.txt) + +- commons-codec:commons-codec:1.15 +- org.apache.hadoop.thirdparty:hadoop-shaded-guava:1.6.0.1-4.3.0-0 +- org.apache.hadoop:hadoop-auth:3.4.3.1-4.3.0-1 +- org.apache.httpcomponents:httpclient:4.5.13 +- org.apache.httpcomponents:httpcore:4.4.14 +- org.apache.kerby:kerb-core:2.0.3 +- org.apache.kerby:kerb-crypto:2.0.3 +- org.apache.kerby:kerb-util:2.0.3 +- org.apache.kerby:kerby-asn1:2.0.3 +- org.apache.kerby:kerby-config:2.0.3 +- org.apache.kerby:kerby-pkix:2.0.3 +- org.apache.kerby:kerby-util:2.0.3 + # Dependency Licenses ## flink-dashboard (2.0.0) ------------------- @@ -3014,4 +3029,3 @@ THE SOFTWARE. --- - diff --git a/flink-runtime-web/src/test/java/org/apache/flink/runtime/webmonitor/history/HistoryServerAuthenticatedUserHandlerTest.java b/flink-runtime-web/src/test/java/org/apache/flink/runtime/webmonitor/history/HistoryServerAuthenticatedUserHandlerTest.java new file mode 100644 index 0000000000000..f3d42daf8acdf --- /dev/null +++ b/flink-runtime-web/src/test/java/org/apache/flink/runtime/webmonitor/history/HistoryServerAuthenticatedUserHandlerTest.java @@ -0,0 +1,85 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.runtime.webmonitor.history; + +import org.apache.flink.runtime.rest.handler.router.RouteResult; +import org.apache.flink.runtime.rest.handler.router.RoutedRequest; + +import org.apache.flink.shaded.netty4.io.netty.buffer.ByteBuf; +import org.apache.flink.shaded.netty4.io.netty.channel.embedded.EmbeddedChannel; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.DefaultFullHttpRequest; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpHeaderNames; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpMethod; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpResponse; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpResponseStatus; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpVersion; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.LastHttpContent; +import org.apache.flink.shaded.netty4.io.netty.util.ReferenceCountUtil; + +import org.junit.jupiter.api.Test; + +import java.nio.charset.StandardCharsets; +import java.util.Collections; + +import static org.assertj.core.api.Assertions.assertThat; + +/** Tests for {@link HistoryServerAuthenticatedUserHandler}. */ +class HistoryServerAuthenticatedUserHandlerTest { + + @Test + void shouldReturnUnauthenticatedResponseWhenAuthenticationIsDisabled() { + EmbeddedChannel channel = new EmbeddedChannel(new HistoryServerAuthenticatedUserHandler()); + DefaultFullHttpRequest request = + new DefaultFullHttpRequest( + HttpVersion.HTTP_1_1, + HttpMethod.GET, + HistoryServerAuthenticatedUserHandler.URL); + + try { + assertThat(channel.writeInbound(routedRequest(request))).isFalse(); + + HttpResponse response = channel.readOutbound(); + assertThat(response.status()).isEqualTo(HttpResponseStatus.OK); + assertThat(response.headers().get(HttpHeaderNames.CACHE_CONTROL)).isEqualTo("no-store"); + ByteBuf content = channel.readOutbound(); + LastHttpContent lastContent = channel.readOutbound(); + try { + assertThat(content.toString(StandardCharsets.UTF_8)) + .contains("\"authenticated\":false"); + assertThat(lastContent).isSameAs(LastHttpContent.EMPTY_LAST_CONTENT); + } finally { + ReferenceCountUtil.release(content); + ReferenceCountUtil.release(lastContent); + } + } finally { + channel.finishAndReleaseAll(); + } + } + + private static RoutedRequest routedRequest(DefaultFullHttpRequest request) { + return new RoutedRequest<>( + new RouteResult<>( + request.uri(), + request.uri(), + Collections.emptyMap(), + Collections.emptyMap(), + new HistoryServerAuthenticatedUserHandler()), + request); + } +} diff --git a/flink-runtime-web/src/test/java/org/apache/flink/runtime/webmonitor/history/HistoryServerSpnegoIntegrationTest.java b/flink-runtime-web/src/test/java/org/apache/flink/runtime/webmonitor/history/HistoryServerSpnegoIntegrationTest.java new file mode 100644 index 0000000000000..eadba73d32b88 --- /dev/null +++ b/flink-runtime-web/src/test/java/org/apache/flink/runtime/webmonitor/history/HistoryServerSpnegoIntegrationTest.java @@ -0,0 +1,299 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.runtime.webmonitor.history; + +import org.apache.flink.api.common.JobID; +import org.apache.flink.api.common.JobStatus; +import org.apache.flink.configuration.Configuration; +import org.apache.flink.configuration.HistoryServerOptions; +import org.apache.flink.configuration.HistoryServerOptions.HistoryServerWebAuthenticationType; +import org.apache.flink.runtime.history.FsJobArchivist; +import org.apache.flink.runtime.rest.messages.JobsOverviewHeaders; +import org.apache.flink.runtime.security.KerberosUtils; +import org.apache.flink.test.util.SecureTestEnvironment; + +import org.apache.hadoop.security.authentication.client.KerberosAuthenticator; +import org.ietf.jgss.GSSContext; +import org.ietf.jgss.GSSManager; +import org.ietf.jgss.GSSName; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.io.TempDir; + +import javax.security.auth.Subject; +import javax.security.auth.callback.CallbackHandler; +import javax.security.auth.login.AppConfigurationEntry; +import javax.security.auth.login.LoginContext; +import javax.security.auth.login.LoginException; + +import java.io.IOException; +import java.io.InputStream; +import java.io.OutputStream; +import java.net.Socket; +import java.net.URL; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.security.PrivilegedExceptionAction; +import java.time.Duration; +import java.util.Base64; +import java.util.Collections; +import java.util.Locale; +import java.util.concurrent.CountDownLatch; +import java.util.concurrent.TimeUnit; + +import static org.assertj.core.api.Assertions.assertThat; + +/** MiniKDC-backed integration tests for HistoryServer SPNEGO authentication. */ +class HistoryServerSpnegoIntegrationTest { + + @TempDir private Path tempDir; + + @Test + void shouldProtectJobsOverviewWithSpnego() throws Exception { + SecureTestEnvironment.prepare(tempDir.resolve("kdc").toFile(), "HTTP/localhost"); + + HistoryServer historyServer = null; + try { + Path archiveDir = Files.createDirectories(tempDir.resolve("archive")); + Path webDir = Files.createDirectories(tempDir.resolve("web")); + JobID jobId = JobID.generate(); + createArchive(archiveDir, jobId); + + CountDownLatch archiveCreated = new CountDownLatch(1); + historyServer = + new HistoryServer( + createHistoryServerConfiguration(archiveDir, webDir), + event -> { + if (event.getType() + == HistoryServerArchiveFetcher.ArchiveEventType.CREATED) { + archiveCreated.countDown(); + } + }); + historyServer.start(); + + assertThat(archiveCreated.await(10L, TimeUnit.SECONDS)).isTrue(); + + URL jobsOverviewUrl = + new URL( + "http://localhost:" + + historyServer.getWebPort() + + JobsOverviewHeaders.URL); + + assertUnauthenticatedRequestIsChallenged(jobsOverviewUrl); + + String response = getWithSpnego(jobsOverviewUrl); + + assertThat(response).contains(jobId.toString()); + } finally { + if (historyServer != null) { + historyServer.stop(); + } + SecureTestEnvironment.cleanup(); + } + } + + private static Configuration createHistoryServerConfiguration(Path archiveDir, Path webDir) { + Configuration configuration = new Configuration(); + configuration.set( + HistoryServerOptions.HISTORY_SERVER_ARCHIVE_DIRS, archiveDir.toUri().toString()); + configuration.set(HistoryServerOptions.HISTORY_SERVER_WEB_DIR, webDir.toString()); + configuration.set( + HistoryServerOptions.HISTORY_SERVER_ARCHIVE_REFRESH_INTERVAL, + Duration.ofMillis(100L)); + configuration.set(HistoryServerOptions.HISTORY_SERVER_WEB_PORT, 0); + configuration.set( + HistoryServerOptions.HISTORY_SERVER_WEB_AUTHENTICATION_TYPE, + HistoryServerWebAuthenticationType.KERBEROS); + configuration.set( + HistoryServerOptions.HISTORY_SERVER_WEB_AUTHENTICATION_KERBEROS_PRINCIPAL, + "HTTP/localhost@" + SecureTestEnvironment.getRealm()); + configuration.set( + HistoryServerOptions.HISTORY_SERVER_WEB_AUTHENTICATION_KERBEROS_KEYTAB, + SecureTestEnvironment.getTestKeytab()); + configuration.set( + HistoryServerOptions.HISTORY_SERVER_WEB_AUTHENTICATION_SIGNATURE_SECRET, + "shared-secret"); + return configuration; + } + + private static void assertUnauthenticatedRequestIsChallenged(URL url) throws IOException { + RawHttpResponse response = getRaw(url, null); + + assertThat(response.statusLine).contains(" 401 "); + assertThat(response.headers.toLowerCase(Locale.ROOT)) + .contains( + KerberosAuthenticator.WWW_AUTHENTICATE.toLowerCase(Locale.ROOT) + + ": " + + KerberosAuthenticator.NEGOTIATE.toLowerCase(Locale.ROOT)); + } + + private static String getWithSpnego(URL url) throws Exception { + Subject clientSubject = loginClientSubject(); + return Subject.doAs( + clientSubject, + (PrivilegedExceptionAction) + () -> { + GSSContext gssContext = null; + try { + GSSManager gssManager = GSSManager.getInstance(); + String servicePrincipal = + org.apache.hadoop.security.authentication.util.KerberosUtil + .getServicePrincipal("HTTP", url.getHost()); + GSSName serviceName = + gssManager.createName( + servicePrincipal, + org.apache.hadoop.security.authentication.util + .KerberosUtil.NT_GSS_KRB5_PRINCIPAL_OID); + gssContext = + gssManager.createContext( + serviceName, + org.apache.hadoop.security.authentication.util + .KerberosUtil.GSS_KRB5_MECH_OID, + null, + GSSContext.DEFAULT_LIFETIME); + gssContext.requestCredDeleg(true); + gssContext.requestMutualAuth(true); + + byte[] outToken = gssContext.initSecContext(new byte[0], 0, 0); + assertThat(outToken).isNotNull(); + + RawHttpResponse response = + getRaw( + url, + KerberosAuthenticator.NEGOTIATE + + " " + + Base64.getEncoder() + .encodeToString(outToken)); + assertThat(response.statusLine).contains(" 200 "); + assertThat(response.headers.toLowerCase(Locale.ROOT)) + .contains("set-cookie: hadoop.auth="); + return response.body; + } finally { + if (gssContext != null) { + gssContext.dispose(); + } + } + }); + } + + private static RawHttpResponse getRaw(URL url, String authorizationHeader) throws IOException { + try (Socket socket = new Socket(url.getHost(), url.getPort())) { + socket.setSoTimeout(10_000); + try (OutputStream outputStream = socket.getOutputStream(); + InputStream inputStream = socket.getInputStream()) { + StringBuilder request = + new StringBuilder() + .append("GET ") + .append(url.getFile()) + .append(" HTTP/1.1\r\nHost: ") + .append(url.getHost()) + .append("\r\nConnection: close\r\n"); + if (authorizationHeader != null) { + request.append(KerberosAuthenticator.AUTHORIZATION) + .append(": ") + .append(authorizationHeader) + .append("\r\n"); + } + request.append("\r\n"); + + outputStream.write(request.toString().getBytes(StandardCharsets.US_ASCII)); + outputStream.flush(); + + String response = new String(inputStream.readAllBytes(), StandardCharsets.UTF_8); + int firstLineEnd = response.indexOf("\r\n"); + int headerEnd = response.indexOf("\r\n\r\n"); + assertThat(firstLineEnd).isGreaterThan(0); + assertThat(headerEnd).isGreaterThan(firstLineEnd); + return new RawHttpResponse( + response.substring(0, firstLineEnd), + response.substring(firstLineEnd + 2, headerEnd), + response.substring(headerEnd + 4)); + } + } + } + + private static Subject loginClientSubject() throws LoginException { + Subject subject = new Subject(); + LoginContext loginContext = + new LoginContext( + "", + subject, + (CallbackHandler) null, + new javax.security.auth.login.Configuration() { + @Override + public AppConfigurationEntry[] getAppConfigurationEntry(String name) { + return new AppConfigurationEntry[] { + KerberosUtils.keytabEntry( + SecureTestEnvironment.getTestKeytab(), + "client/localhost@" + SecureTestEnvironment.getRealm()) + }; + } + }); + loginContext.login(); + return subject; + } + + private static void createArchive(Path archiveDir, JobID jobId) throws IOException { + ArchivedJson archivedJson = new ArchivedJson("/joboverview", createJobOverviewJson(jobId)); + FsJobArchivist.archiveJob( + new org.apache.flink.core.fs.Path(archiveDir.toUri()), + jobId, + Collections.singleton(archivedJson)); + } + + private static String createJobOverviewJson(JobID jobId) { + return "{" + + "\"finished\":[{" + + "\"jid\":\"" + + jobId + + "\"," + + "\"name\":\"spnego-test\"," + + "\"state\":\"" + + JobStatus.FINISHED.name() + + "\"," + + "\"start-time\":0," + + "\"end-time\":1," + + "\"duration\":1," + + "\"last-modification\":1," + + "\"tasks\":{" + + "\"total\":0," + + "\"created\":0," + + "\"deploying\":0," + + "\"scheduled\":0," + + "\"running\":0," + + "\"finished\":0," + + "\"canceling\":0," + + "\"canceled\":0," + + "\"failed\":0" + + "}}]}"; + } + + private static final class RawHttpResponse { + + private final String statusLine; + private final String headers; + private final String body; + + private RawHttpResponse(String statusLine, String headers, String body) { + this.statusLine = statusLine; + this.headers = headers; + this.body = body; + } + } +} diff --git a/flink-runtime-web/src/test/java/org/apache/flink/runtime/webmonitor/history/HistoryServerTest.java b/flink-runtime-web/src/test/java/org/apache/flink/runtime/webmonitor/history/HistoryServerTest.java index 9ab2bb95f8f81..82ee11fcf6096 100644 --- a/flink-runtime-web/src/test/java/org/apache/flink/runtime/webmonitor/history/HistoryServerTest.java +++ b/flink-runtime-web/src/test/java/org/apache/flink/runtime/webmonitor/history/HistoryServerTest.java @@ -36,7 +36,6 @@ import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment; import org.apache.flink.streaming.api.functions.sink.v2.DiscardingSink; import org.apache.flink.test.util.MiniClusterWithClientResource; -import org.apache.flink.util.FlinkException; import org.apache.flink.util.jackson.JacksonMapperFactory; import org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.JsonFactory; @@ -251,8 +250,7 @@ void testFailIfHistorySizeLimitIsLessThanMinusOne() throws Exception { .isInstanceOf(IllegalConfigurationException.class); } - private void startHistoryServerWithSizeLimit(int maxHistorySize) - throws IOException, FlinkException, InterruptedException { + private void startHistoryServerWithSizeLimit(int maxHistorySize) throws Exception { Configuration historyServerConfig = createTestConfiguration( HistoryServerOptions.HISTORY_SERVER_CLEANUP_EXPIRED_JOBS.defaultValue()); diff --git a/flink-runtime-web/src/test/java/org/apache/flink/runtime/webmonitor/history/security/HistoryServerSpnegoConfigTest.java b/flink-runtime-web/src/test/java/org/apache/flink/runtime/webmonitor/history/security/HistoryServerSpnegoConfigTest.java new file mode 100644 index 0000000000000..2654c35a8f1e6 --- /dev/null +++ b/flink-runtime-web/src/test/java/org/apache/flink/runtime/webmonitor/history/security/HistoryServerSpnegoConfigTest.java @@ -0,0 +1,202 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.runtime.webmonitor.history.security; + +import org.apache.flink.configuration.Configuration; +import org.apache.flink.configuration.HistoryServerOptions; +import org.apache.flink.configuration.HistoryServerOptions.HistoryServerWebAuthenticationType; +import org.apache.flink.util.ConfigurationException; + +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.io.TempDir; + +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.time.Duration; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.assertThatThrownBy; + +/** Tests for {@link HistoryServerWebAuthenticationConfig}. */ +class HistoryServerSpnegoConfigTest { + + @TempDir private Path tempDir; + + @Test + void shouldReturnEmptyConfigWhenAuthenticationIsDisabled() throws Exception { + assertThat(HistoryServerWebAuthenticationConfig.from(new Configuration())).isEmpty(); + } + + @Test + void shouldRequirePrincipalWhenKerberosIsEnabled() throws Exception { + Configuration configuration = kerberosConfiguration(); + configuration.set( + HistoryServerOptions.HISTORY_SERVER_WEB_AUTHENTICATION_KERBEROS_KEYTAB, + createKeytab().toString()); + + assertThatThrownBy(() -> HistoryServerWebAuthenticationConfig.from(configuration)) + .isInstanceOf(ConfigurationException.class) + .hasMessageContaining( + HistoryServerOptions.HISTORY_SERVER_WEB_AUTHENTICATION_KERBEROS_PRINCIPAL + .key()); + } + + @Test + void shouldRejectInvalidAuthenticationType() { + Configuration configuration = new Configuration(); + configuration.setString( + HistoryServerOptions.HISTORY_SERVER_WEB_AUTHENTICATION_TYPE.key(), "BASIC"); + + assertThatThrownBy(() -> HistoryServerWebAuthenticationConfig.from(configuration)) + .isInstanceOf(ConfigurationException.class) + .hasMessageContaining( + HistoryServerOptions.HISTORY_SERVER_WEB_AUTHENTICATION_TYPE.key()); + } + + @Test + void shouldRequireKeytabWhenKerberosIsEnabled() { + Configuration configuration = kerberosConfiguration(); + configuration.set( + HistoryServerOptions.HISTORY_SERVER_WEB_AUTHENTICATION_KERBEROS_PRINCIPAL, + "HTTP/localhost@EXAMPLE.COM"); + + assertThatThrownBy(() -> HistoryServerWebAuthenticationConfig.from(configuration)) + .isInstanceOf(ConfigurationException.class) + .hasMessageContaining( + HistoryServerOptions.HISTORY_SERVER_WEB_AUTHENTICATION_KERBEROS_KEYTAB + .key()); + } + + @Test + void shouldResolveHostPlaceholderInPrincipal() throws Exception { + Configuration configuration = kerberosConfiguration(); + configuration.set( + HistoryServerOptions.HISTORY_SERVER_WEB_AUTHENTICATION_KERBEROS_PRINCIPAL, + "HTTP/_HOST@EXAMPLE.COM"); + configuration.set( + HistoryServerOptions.HISTORY_SERVER_WEB_AUTHENTICATION_KERBEROS_KEYTAB, + createKeytab().toString()); + + HistoryServerWebAuthenticationConfig config = + HistoryServerWebAuthenticationConfig.from(configuration).orElseThrow(); + + assertThat(config.getKerberosPrincipal()) + .startsWith("HTTP/") + .endsWith("@EXAMPLE.COM") + .doesNotContain("_HOST"); + } + + @Test + void shouldRejectNonHttpPrincipal() throws Exception { + Configuration configuration = kerberosConfiguration(); + configuration.set( + HistoryServerOptions.HISTORY_SERVER_WEB_AUTHENTICATION_KERBEROS_PRINCIPAL, + "flink/localhost@EXAMPLE.COM"); + configuration.set( + HistoryServerOptions.HISTORY_SERVER_WEB_AUTHENTICATION_KERBEROS_KEYTAB, + createKeytab().toString()); + + assertThatThrownBy(() -> HistoryServerWebAuthenticationConfig.from(configuration)) + .isInstanceOf(ConfigurationException.class) + .hasMessageContaining("must start with HTTP/"); + } + + @Test + void shouldRejectInvalidCookiePath() throws Exception { + Configuration configuration = validKerberosConfiguration(); + configuration.set( + HistoryServerOptions.HISTORY_SERVER_WEB_AUTHENTICATION_COOKIE_PATH, "history"); + + assertThatThrownBy(() -> HistoryServerWebAuthenticationConfig.from(configuration)) + .isInstanceOf(ConfigurationException.class) + .hasMessageContaining( + HistoryServerOptions.HISTORY_SERVER_WEB_AUTHENTICATION_COOKIE_PATH.key()); + } + + @Test + void shouldRejectNonPositiveTokenValidity() throws Exception { + Configuration configuration = validKerberosConfiguration(); + configuration.set( + HistoryServerOptions.HISTORY_SERVER_WEB_AUTHENTICATION_TOKEN_VALIDITY, + Duration.ZERO); + + assertThatThrownBy(() -> HistoryServerWebAuthenticationConfig.from(configuration)) + .isInstanceOf(ConfigurationException.class) + .hasMessageContaining( + HistoryServerOptions.HISTORY_SERVER_WEB_AUTHENTICATION_TOKEN_VALIDITY + .key()); + } + + @Test + void shouldRejectMultipleSignatureSecretSources() throws Exception { + Configuration configuration = validKerberosConfiguration(); + configuration.set( + HistoryServerOptions.HISTORY_SERVER_WEB_AUTHENTICATION_SIGNATURE_SECRET, "secret"); + configuration.set( + HistoryServerOptions.HISTORY_SERVER_WEB_AUTHENTICATION_SIGNATURE_SECRET_FILE, + createSecretFile("secret").toString()); + + assertThatThrownBy(() -> HistoryServerWebAuthenticationConfig.from(configuration)) + .isInstanceOf(ConfigurationException.class) + .hasMessageContaining("Only one of"); + } + + @Test + void shouldLoadSignatureSecretFromFile() throws Exception { + Configuration configuration = validKerberosConfiguration(); + configuration.set( + HistoryServerOptions.HISTORY_SERVER_WEB_AUTHENTICATION_SIGNATURE_SECRET_FILE, + createSecretFile("shared-secret\n").toString()); + + HistoryServerWebAuthenticationConfig config = + HistoryServerWebAuthenticationConfig.from(configuration).orElseThrow(); + + assertThat(new String(config.getSignatureSecret(), StandardCharsets.UTF_8)) + .isEqualTo("shared-secret"); + } + + private Configuration validKerberosConfiguration() throws Exception { + Configuration configuration = kerberosConfiguration(); + configuration.set( + HistoryServerOptions.HISTORY_SERVER_WEB_AUTHENTICATION_KERBEROS_PRINCIPAL, + "HTTP/localhost@EXAMPLE.COM"); + configuration.set( + HistoryServerOptions.HISTORY_SERVER_WEB_AUTHENTICATION_KERBEROS_KEYTAB, + createKeytab().toString()); + return configuration; + } + + private Configuration kerberosConfiguration() { + Configuration configuration = new Configuration(); + configuration.set( + HistoryServerOptions.HISTORY_SERVER_WEB_AUTHENTICATION_TYPE, + HistoryServerWebAuthenticationType.KERBEROS); + return configuration; + } + + private Path createKeytab() throws Exception { + return Files.createFile(tempDir.resolve("test-" + System.nanoTime() + ".keytab")); + } + + private Path createSecretFile(String secret) throws Exception { + Path secretFile = tempDir.resolve("secret-" + System.nanoTime()); + return Files.write(secretFile, secret.getBytes(StandardCharsets.UTF_8)); + } +} diff --git a/flink-runtime-web/src/test/java/org/apache/flink/runtime/webmonitor/history/security/HistoryServerSpnegoHandlerTest.java b/flink-runtime-web/src/test/java/org/apache/flink/runtime/webmonitor/history/security/HistoryServerSpnegoHandlerTest.java new file mode 100644 index 0000000000000..0e8e628a3eb6c --- /dev/null +++ b/flink-runtime-web/src/test/java/org/apache/flink/runtime/webmonitor/history/security/HistoryServerSpnegoHandlerTest.java @@ -0,0 +1,326 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.runtime.webmonitor.history.security; + +import org.apache.flink.configuration.Configuration; +import org.apache.flink.configuration.HistoryServerOptions; +import org.apache.flink.configuration.HistoryServerOptions.HistoryServerWebAuthenticationType; + +import org.apache.flink.shaded.netty4.io.netty.channel.ChannelHandlerContext; +import org.apache.flink.shaded.netty4.io.netty.channel.ChannelInboundHandlerAdapter; +import org.apache.flink.shaded.netty4.io.netty.channel.embedded.EmbeddedChannel; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.DefaultFullHttpRequest; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.DefaultHttpRequest; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.DefaultLastHttpContent; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.FullHttpRequest; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.FullHttpResponse; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpHeaderNames; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpMethod; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpRequest; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpResponseStatus; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpVersion; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.LastHttpContent; +import org.apache.flink.shaded.netty4.io.netty.util.ReferenceCountUtil; + +import org.apache.hadoop.security.authentication.client.AuthenticatedURL; +import org.apache.hadoop.security.authentication.client.KerberosAuthenticator; +import org.apache.hadoop.security.authentication.server.AuthenticationToken; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.io.TempDir; + +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.Optional; +import java.util.concurrent.atomic.AtomicReference; + +import static org.assertj.core.api.Assertions.assertThat; + +/** Tests for {@link HistoryServerWebAuthenticationHandler}. */ +class HistoryServerSpnegoHandlerTest { + + private static final byte[] SECRET = "shared-secret".getBytes(StandardCharsets.UTF_8); + + @TempDir private Path tempDir; + + @Test + void shouldNotCreateHandlerWhenAuthenticationIsDisabled() throws Exception { + assertThat(HistoryServerWebAuthenticationHandler.createFactory(new Configuration(), false)) + .isEmpty(); + } + + @Test + void shouldChallengeUnauthenticatedRequest() throws Exception { + EmbeddedChannel channel = createChannel(false); + try { + assertThat(channel.writeInbound(getRequest())).isFalse(); + + FullHttpResponse response = channel.readOutbound(); + assertThat(response.status()).isEqualTo(HttpResponseStatus.UNAUTHORIZED); + assertThat(response.headers().get(HttpHeaderNames.WWW_AUTHENTICATE)) + .isEqualTo(KerberosAuthenticator.NEGOTIATE); + assertThat(response.headers().getInt(HttpHeaderNames.CONTENT_LENGTH)).isZero(); + assertThat(response.headers().getAll(HttpHeaderNames.SET_COOKIE)) + .anySatisfy( + cookie -> + assertThat(cookie) + .startsWith(AuthenticatedURL.AUTH_COOKIE + "=") + .contains("Max-Age=0") + .contains("HttpOnly")); + response.release(); + } finally { + channel.finishAndReleaseAll(); + } + } + + @Test + void shouldChallengeUnauthenticatedStreamingRequest() throws Exception { + EmbeddedChannel channel = createChannel(false); + try { + HttpRequest request = + new DefaultHttpRequest(HttpVersion.HTTP_1_1, HttpMethod.GET, "/jobs/overview"); + + assertThat(channel.writeInbound(request)).isFalse(); + + FullHttpResponse response = channel.readOutbound(); + assertThat(response.status()).isEqualTo(HttpResponseStatus.UNAUTHORIZED); + assertThat(response.headers().get(HttpHeaderNames.WWW_AUTHENTICATE)) + .isEqualTo(KerberosAuthenticator.NEGOTIATE); + assertThat((Object) channel.readInbound()).isNull(); + response.release(); + } finally { + channel.finishAndReleaseAll(); + } + } + + @Test + void shouldRejectMalformedSpnegoToken() throws Exception { + EmbeddedChannel channel = createChannel(false); + try { + FullHttpRequest request = getRequest(); + request.headers() + .set( + KerberosAuthenticator.AUTHORIZATION, + KerberosAuthenticator.NEGOTIATE + " %%%"); + + assertThat(channel.writeInbound(request)).isFalse(); + + FullHttpResponse response = channel.readOutbound(); + assertThat(response.status()).isEqualTo(HttpResponseStatus.FORBIDDEN); + assertThat(response.headers().contains(HttpHeaderNames.WWW_AUTHENTICATE)).isFalse(); + assertThat(response.headers().getAll(HttpHeaderNames.SET_COOKIE)) + .anySatisfy( + cookie -> + assertThat(cookie) + .startsWith(AuthenticatedURL.AUTH_COOKIE + "=") + .contains("Max-Age=0")); + response.release(); + } finally { + channel.finishAndReleaseAll(); + } + } + + @Test + void shouldPassThroughRequestWithValidSignedCookie() throws Exception { + EmbeddedChannel channel = createChannel(false); + FullHttpRequest request = getRequest(); + request.headers() + .set( + HttpHeaderNames.COOKIE, + AuthenticatedURL.AUTH_COOKIE + + "=\"" + + createSignedCookie(System.currentTimeMillis() + 60_000L) + + "\""); + + try { + assertThat(channel.writeInbound(request)).isTrue(); + assertThat((Object) channel.readInbound()).isSameAs(request); + } finally { + request.release(); + channel.finishAndReleaseAll(); + } + } + + @Test + void shouldExposeAuthenticatedUserWhileHandlingRequestWithValidSignedCookie() throws Exception { + AtomicReference> authenticatedUser = + new AtomicReference<>(Optional.empty()); + EmbeddedChannel channel = + createChannel( + false, + new ChannelInboundHandlerAdapter() { + @Override + public void channelRead(ChannelHandlerContext ctx, Object msg) { + authenticatedUser.set( + HistoryServerWebAuthenticationHandler.getAuthenticatedUser( + ctx)); + ReferenceCountUtil.release(msg); + } + }); + FullHttpRequest request = getRequest(); + request.headers() + .set( + HttpHeaderNames.COOKIE, + AuthenticatedURL.AUTH_COOKIE + + "=\"" + + createSignedCookie(System.currentTimeMillis() + 60_000L) + + "\""); + + try { + assertThat(channel.writeInbound(request)).isFalse(); + assertThat(authenticatedUser.get()) + .hasValueSatisfying( + user -> { + assertThat(user.getUserName()).isEqualTo("alice"); + assertThat(user.getPrincipal()).isEqualTo("alice@EXAMPLE.COM"); + assertThat(user.getType()) + .isEqualTo(HistoryServerSpnegoAuthenticator.TOKEN_TYPE); + }); + } finally { + channel.finishAndReleaseAll(); + } + } + + @Test + void shouldKeepAuthenticatedUserUntilEndOfStreamingRequest() throws Exception { + AtomicReference> userDuringRequest = + new AtomicReference<>(Optional.empty()); + AtomicReference> userDuringLastContent = + new AtomicReference<>(Optional.empty()); + EmbeddedChannel channel = + createChannel( + false, + new ChannelInboundHandlerAdapter() { + @Override + public void channelRead(ChannelHandlerContext ctx, Object msg) { + if (msg instanceof HttpRequest) { + userDuringRequest.set( + HistoryServerWebAuthenticationHandler + .getAuthenticatedUser(ctx)); + } else if (msg instanceof LastHttpContent) { + userDuringLastContent.set( + HistoryServerWebAuthenticationHandler + .getAuthenticatedUser(ctx)); + } + ReferenceCountUtil.release(msg); + } + }); + HttpRequest request = + new DefaultHttpRequest(HttpVersion.HTTP_1_1, HttpMethod.GET, "/auth/user"); + request.headers() + .set( + HttpHeaderNames.COOKIE, + AuthenticatedURL.AUTH_COOKIE + + "=\"" + + createSignedCookie(System.currentTimeMillis() + 60_000L) + + "\""); + + try { + assertThat(channel.writeInbound(request)).isFalse(); + assertThat(channel.writeInbound(new DefaultLastHttpContent())).isFalse(); + + assertThat(userDuringRequest.get()).isPresent(); + assertThat(userDuringLastContent.get()).isPresent(); + } finally { + channel.finishAndReleaseAll(); + } + } + + @Test + void shouldRestartSpnegoChallengeForExpiredCookie() throws Exception { + EmbeddedChannel channel = createChannel(false); + FullHttpRequest request = getRequest(); + request.headers() + .set( + HttpHeaderNames.COOKIE, + AuthenticatedURL.AUTH_COOKIE + + "=\"" + + createSignedCookie(System.currentTimeMillis() - 60_000L) + + "\""); + + try { + assertThat(channel.writeInbound(request)).isFalse(); + + FullHttpResponse response = channel.readOutbound(); + assertThat(response.status()).isEqualTo(HttpResponseStatus.UNAUTHORIZED); + assertThat(response.headers().get(HttpHeaderNames.WWW_AUTHENTICATE)) + .isEqualTo(KerberosAuthenticator.NEGOTIATE); + response.release(); + } finally { + channel.finishAndReleaseAll(); + } + } + + @Test + void shouldMarkAuthenticationCookieSecureWhenSslIsEnabled() throws Exception { + EmbeddedChannel channel = createChannel(true); + try { + assertThat(channel.writeInbound(getRequest())).isFalse(); + + FullHttpResponse response = channel.readOutbound(); + assertThat(response.headers().getAll(HttpHeaderNames.SET_COOKIE)) + .anySatisfy(cookie -> assertThat(cookie).contains("Secure")); + response.release(); + } finally { + channel.finishAndReleaseAll(); + } + } + + private EmbeddedChannel createChannel(boolean secureCookie) throws Exception { + return createChannel(secureCookie, new ChannelInboundHandlerAdapter()); + } + + private EmbeddedChannel createChannel( + boolean secureCookie, ChannelInboundHandlerAdapter tailHandler) throws Exception { + HistoryServerWebAuthenticationHandler.Factory factory = + HistoryServerWebAuthenticationHandler.createFactory( + createKerberosConfiguration(), secureCookie) + .orElseThrow(); + return new EmbeddedChannel(factory.createHandler(), tailHandler); + } + + private Configuration createKerberosConfiguration() throws Exception { + Configuration configuration = new Configuration(); + configuration.set( + HistoryServerOptions.HISTORY_SERVER_WEB_AUTHENTICATION_TYPE, + HistoryServerWebAuthenticationType.KERBEROS); + configuration.set( + HistoryServerOptions.HISTORY_SERVER_WEB_AUTHENTICATION_KERBEROS_PRINCIPAL, + "HTTP/localhost@EXAMPLE.COM"); + configuration.set( + HistoryServerOptions.HISTORY_SERVER_WEB_AUTHENTICATION_KERBEROS_KEYTAB, + Files.createTempFile(tempDir, "history-server", ".keytab").toString()); + configuration.set( + HistoryServerOptions.HISTORY_SERVER_WEB_AUTHENTICATION_SIGNATURE_SECRET, + new String(SECRET, StandardCharsets.UTF_8)); + return configuration; + } + + private static FullHttpRequest getRequest() { + return new DefaultFullHttpRequest(HttpVersion.HTTP_1_1, HttpMethod.GET, "/jobs/overview"); + } + + private static String createSignedCookie(long expires) { + AuthenticationToken token = + new AuthenticationToken( + "alice", "alice@EXAMPLE.COM", HistoryServerSpnegoAuthenticator.TOKEN_TYPE); + token.setExpires(expires); + return new HistoryServerAuthenticationTokenSigner(SECRET).sign(token); + } +} diff --git a/flink-runtime-web/src/test/java/org/apache/flink/runtime/webmonitor/history/security/HistoryServerSpnegoTokenSignerTest.java b/flink-runtime-web/src/test/java/org/apache/flink/runtime/webmonitor/history/security/HistoryServerSpnegoTokenSignerTest.java new file mode 100644 index 0000000000000..d6cc2e6751d20 --- /dev/null +++ b/flink-runtime-web/src/test/java/org/apache/flink/runtime/webmonitor/history/security/HistoryServerSpnegoTokenSignerTest.java @@ -0,0 +1,123 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.runtime.webmonitor.history.security; + +import org.apache.hadoop.security.authentication.client.AuthenticationException; +import org.apache.hadoop.security.authentication.server.AuthenticationToken; +import org.apache.hadoop.security.authentication.util.Signer; +import org.apache.hadoop.security.authentication.util.SignerSecretProvider; +import org.junit.jupiter.api.Test; + +import javax.servlet.ServletContext; + +import java.nio.charset.StandardCharsets; +import java.util.Properties; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.assertThatThrownBy; + +/** Tests for {@link HistoryServerAuthenticationTokenSigner}. */ +class HistoryServerSpnegoTokenSignerTest { + + private static final byte[] SECRET = "shared-secret".getBytes(StandardCharsets.UTF_8); + + @Test + void shouldProduceHadoopAuthCompatibleSignature() throws Exception { + AuthenticationToken token = createToken(System.currentTimeMillis() + 60_000L); + + String signedToken = new HistoryServerAuthenticationTokenSigner(SECRET).sign(token); + String hadoopSignedToken = + new Signer(new StaticSecretProvider(SECRET)).sign(token.toString()); + + assertThat(signedToken).isEqualTo(hadoopSignedToken); + assertThat(new Signer(new StaticSecretProvider(SECRET)).verifyAndExtract(signedToken)) + .isEqualTo(token.toString()); + } + + @Test + void shouldVerifySignedToken() throws Exception { + HistoryServerAuthenticationTokenSigner signer = + new HistoryServerAuthenticationTokenSigner(SECRET); + AuthenticationToken token = createToken(System.currentTimeMillis() + 60_000L); + + AuthenticationToken verifiedToken = signer.verifyAndExtract(signer.sign(token)); + + assertThat(verifiedToken.getUserName()).isEqualTo("alice"); + assertThat(verifiedToken.getName()).isEqualTo("alice@EXAMPLE.COM"); + assertThat(verifiedToken.getType()).isEqualTo(HistoryServerSpnegoAuthenticator.TOKEN_TYPE); + assertThat(verifiedToken.isExpired()).isFalse(); + } + + @Test + void shouldRejectTamperedToken() { + HistoryServerAuthenticationTokenSigner signer = + new HistoryServerAuthenticationTokenSigner(SECRET); + String signedToken = signer.sign(createToken(System.currentTimeMillis() + 60_000L)); + + assertThatThrownBy(() -> signer.verifyAndExtract(signedToken + "tampered")) + .isInstanceOf(AuthenticationException.class); + } + + @Test + void shouldStripCookieQuotesBeforeVerification() throws Exception { + HistoryServerAuthenticationTokenSigner signer = + new HistoryServerAuthenticationTokenSigner(SECRET); + String signedToken = signer.sign(createToken(System.currentTimeMillis() + 60_000L)); + + assertThat(signer.verifyAndExtract("\"" + signedToken + "\"").getUserName()) + .isEqualTo("alice"); + } + + @Test + void shouldRejectEmptySecret() { + assertThatThrownBy(() -> new HistoryServerAuthenticationTokenSigner(new byte[0])) + .isInstanceOf(IllegalArgumentException.class) + .hasMessageContaining("secret"); + } + + private static AuthenticationToken createToken(long expires) { + AuthenticationToken token = + new AuthenticationToken( + "alice", "alice@EXAMPLE.COM", HistoryServerSpnegoAuthenticator.TOKEN_TYPE); + token.setExpires(expires); + return token; + } + + private static final class StaticSecretProvider extends SignerSecretProvider { + + private final byte[] secret; + + private StaticSecretProvider(byte[] secret) { + this.secret = secret.clone(); + } + + @Override + public void init(Properties config, ServletContext servletContext, long tokenValidity) {} + + @Override + public byte[] getCurrentSecret() { + return secret.clone(); + } + + @Override + public byte[][] getAllSecrets() { + return new byte[][] {secret.clone()}; + } + } +} diff --git a/flink-runtime-web/web-dashboard/src/app/app.component.html b/flink-runtime-web/web-dashboard/src/app/app.component.html index b14ff6f4693a5..11959b680b3b7 100644 --- a/flink-runtime-web/web-dashboard/src/app/app.component.html +++ b/flink-runtime-web/web-dashboard/src/app/app.component.html @@ -99,6 +99,18 @@

Apache Flink Dashboard

Commit: {{ statusService.configuration['flink-revision'] }} + + + + + User: + {{ authenticatedUser.user }} + + + Message: diff --git a/flink-runtime-web/web-dashboard/src/app/app.component.ts b/flink-runtime-web/web-dashboard/src/app/app.component.ts index f7a21b2ecaee4..272434c662e52 100644 --- a/flink-runtime-web/web-dashboard/src/app/app.component.ts +++ b/flink-runtime-web/web-dashboard/src/app/app.component.ts @@ -17,12 +17,13 @@ */ import { AsyncPipe, NgForOf, NgIf } from '@angular/common'; +import { HttpClient } from '@angular/common/http'; import { ChangeDetectionStrategy, ChangeDetectorRef, Component } from '@angular/core'; import { RouterLink, RouterLinkActive, RouterOutlet } from '@angular/router'; -import { fromEvent, merge } from 'rxjs'; -import { map, startWith } from 'rxjs/operators'; +import { fromEvent, merge, Observable, of } from 'rxjs'; +import { catchError, map, shareReplay, startWith } from 'rxjs/operators'; -import { StatusService } from '@flink-runtime-web/services'; +import { ConfigService, StatusService } from '@flink-runtime-web/services'; import { NzAlertModule } from 'ng-zorro-antd/alert'; import { NzBadgeModule } from 'ng-zorro-antd/badge'; import { NzDividerModule } from 'ng-zorro-antd/divider'; @@ -31,6 +32,13 @@ import { NzIconModule } from 'ng-zorro-antd/icon'; import { NzLayoutModule } from 'ng-zorro-antd/layout'; import { NzMenuModule } from 'ng-zorro-antd/menu'; +interface AuthenticatedUser { + authenticated: boolean; + user?: string; + principal?: string; + type?: string; +} + @Component({ selector: 'flink-root', templateUrl: './app.component.html', @@ -55,6 +63,7 @@ import { NzMenuModule } from 'ng-zorro-antd/menu'; export class AppComponent { collapsed = false; visible = false; + authenticatedUser$: Observable; online$ = merge( fromEvent(window, 'offline').pipe(map(() => false)), fromEvent(window, 'online').pipe(map(() => true)) @@ -81,5 +90,15 @@ export class AppComponent { this.cdr.markForCheck(); } - constructor(public statusService: StatusService, private cdr: ChangeDetectorRef) {} + constructor( + public statusService: StatusService, + private cdr: ChangeDetectorRef, + private httpClient: HttpClient, + private configService: ConfigService + ) { + this.authenticatedUser$ = this.httpClient.get(`${this.configService.BASE_URL}/auth/user`).pipe( + catchError(() => of({ authenticated: false })), + shareReplay({ bufferSize: 1, refCount: true }) + ); + } } diff --git a/flink-runtime/pom.xml b/flink-runtime/pom.xml index fed66a0ec26bf..fe1d9792e5e24 100644 --- a/flink-runtime/pom.xml +++ b/flink-runtime/pom.xml @@ -172,6 +172,47 @@ under the License. + + org.apache.hadoop + hadoop-auth + ${flink.hadoop.version} + ${flink.markBundledAsOptional} + + + log4j + log4j + + + org.slf4j + slf4j-log4j12 + + + ch.qos.reload4j + reload4j + + + org.slf4j + slf4j-reload4j + + + io.dropwizard.metrics + metrics-core + + + org.apache.zookeeper + zookeeper + + + org.apache.curator + curator-framework + + + io.netty + netty-handler + + + + net.jcip @@ -326,6 +367,31 @@ under the License. mockito-subclass test + + + org.apache.hadoop + hadoop-minikdc + ${minikdc.version} + test + + + log4j + log4j + + + org.slf4j + slf4j-log4j12 + + + ch.qos.reload4j + reload4j + + + org.slf4j + slf4j-reload4j + + + @@ -434,8 +500,20 @@ under the License. - + io.airlift:aircompressor + org.apache.hadoop:hadoop-auth + commons-codec:commons-codec + org.apache.httpcomponents:httpclient + org.apache.httpcomponents:httpcore + org.apache.hadoop.thirdparty:hadoop-shaded-guava + org.apache.kerby:kerb-core + org.apache.kerby:kerby-pkix + org.apache.kerby:kerby-asn1 + org.apache.kerby:kerby-util + org.apache.kerby:kerb-util + org.apache.kerby:kerby-config + org.apache.kerby:kerb-crypto @@ -445,6 +523,18 @@ under the License. org.apache.flink.shaded.io.airlift.compress + + org.apache.hadoop.security.authentication + + org.apache.flink.runtime.shaded.org.apache.hadoop.security.authentication + + + + org.apache.hadoop.thirdparty + + org.apache.flink.runtime.shaded.org.apache.hadoop.thirdparty + + diff --git a/flink-runtime/src/main/java/org/apache/flink/runtime/rest/RestServerEndpoint.java b/flink-runtime/src/main/java/org/apache/flink/runtime/rest/RestServerEndpoint.java index bd521227b8c11..221d911725329 100644 --- a/flink-runtime/src/main/java/org/apache/flink/runtime/rest/RestServerEndpoint.java +++ b/flink-runtime/src/main/java/org/apache/flink/runtime/rest/RestServerEndpoint.java @@ -169,6 +169,15 @@ List getInboundChannelHandlerFactories() { protected abstract List> initializeHandlers(final CompletableFuture localAddressFuture); + /** + * Creates endpoint-specific Netty handlers that run after request aggregation and before + * service-loaded inbound handlers and routing. + */ + protected Collection createEndpointSpecificChannelHandlers() + throws ConfigurationException { + return Collections.emptyList(); + } + /** * Starts this REST server endpoint. * @@ -226,6 +235,11 @@ protected void initChannel(SocketChannel ch) throws ConfigurationException { new FlinkHttpObjectAggregator( maxContentLength, responseHeaders)); + for (ChannelHandler channelHandler : + createEndpointSpecificChannelHandlers()) { + ch.pipeline().addLast(channelHandler); + } + for (InboundChannelHandlerFactory factory : inboundChannelHandlerFactories) { Optional channelHandler = diff --git a/flink-runtime/src/main/java/org/apache/flink/runtime/webmonitor/WebMonitorEndpoint.java b/flink-runtime/src/main/java/org/apache/flink/runtime/webmonitor/WebMonitorEndpoint.java index c54631c8faf35..294d052a776f4 100644 --- a/flink-runtime/src/main/java/org/apache/flink/runtime/webmonitor/WebMonitorEndpoint.java +++ b/flink-runtime/src/main/java/org/apache/flink/runtime/webmonitor/WebMonitorEndpoint.java @@ -166,6 +166,9 @@ import org.apache.flink.runtime.webmonitor.history.ArchivedJson; import org.apache.flink.runtime.webmonitor.history.JsonArchivist; import org.apache.flink.runtime.webmonitor.retriever.GatewayRetriever; +import org.apache.flink.runtime.webmonitor.security.JobManagerAuthenticatedUserHandler; +import org.apache.flink.runtime.webmonitor.security.JobManagerAuthenticatedUserHeaders; +import org.apache.flink.runtime.webmonitor.security.JobManagerWebAuthenticationHandler; import org.apache.flink.runtime.webmonitor.threadinfo.ThreadInfoRequestCoordinator; import org.apache.flink.runtime.webmonitor.threadinfo.VertexThreadInfoTracker; import org.apache.flink.runtime.webmonitor.threadinfo.VertexThreadInfoTrackerBuilder; @@ -179,6 +182,7 @@ import org.apache.flink.shaded.guava33.com.google.common.cache.Cache; import org.apache.flink.shaded.guava33.com.google.common.cache.CacheBuilder; +import org.apache.flink.shaded.netty4.io.netty.channel.ChannelHandler; import org.apache.flink.shaded.netty4.io.netty.channel.ChannelInboundHandler; import javax.annotation.Nullable; @@ -228,6 +232,9 @@ public class WebMonitorEndpoint extends RestServerEndp private final Collection archivingHandlers = new ArrayList<>(16); + private final Optional + webAuthenticationHandlerFactory; + @Nullable private ScheduledFuture executionGraphCleanupTask; public WebMonitorEndpoint( @@ -264,6 +271,17 @@ public WebMonitorEndpoint( this.leaderElection = Preconditions.checkNotNull(leaderElection); this.fatalErrorHandler = Preconditions.checkNotNull(fatalErrorHandler); + this.webAuthenticationHandlerFactory = + JobManagerWebAuthenticationHandler.createFactory( + this.clusterConfiguration, responseHeaders); + } + + @Override + protected Collection createEndpointSpecificChannelHandlers() { + return webAuthenticationHandlerFactory + .>map( + factory -> Collections.singletonList(factory.createHandler())) + .orElseGet(Collections::emptyList); } private VertexThreadInfoTracker initializeThreadInfoTracker(ScheduledExecutorService executor) { @@ -320,6 +338,9 @@ protected List> initiali restConfiguration.isWebCancelEnabled(), restConfiguration.isWebRescaleEnabled()); + JobManagerAuthenticatedUserHandler authenticatedUserHandler = + new JobManagerAuthenticatedUserHandler(); + JobIdsHandler jobIdsHandler = new JobIdsHandler( leaderRetriever, @@ -750,6 +771,10 @@ protected List> initiali jobManagerJobEnvironmentHandler.getMessageHeaders(), jobManagerJobEnvironmentHandler)); handlers.add(Tuple2.of(dashboardConfigHandler.getMessageHeaders(), dashboardConfigHandler)); + handlers.add( + Tuple2.of( + JobManagerAuthenticatedUserHeaders.getInstance(), + authenticatedUserHandler)); handlers.add(Tuple2.of(jobIdsHandler.getMessageHeaders(), jobIdsHandler)); handlers.add(Tuple2.of(jobStatusHandler.getMessageHeaders(), jobStatusHandler)); handlers.add(Tuple2.of(jobsOverviewHandler.getMessageHeaders(), jobsOverviewHandler)); diff --git a/flink-runtime/src/main/java/org/apache/flink/runtime/webmonitor/security/JobManagerAuthenticatedUser.java b/flink-runtime/src/main/java/org/apache/flink/runtime/webmonitor/security/JobManagerAuthenticatedUser.java new file mode 100644 index 0000000000000..6012bdc6bcdaa --- /dev/null +++ b/flink-runtime/src/main/java/org/apache/flink/runtime/webmonitor/security/JobManagerAuthenticatedUser.java @@ -0,0 +1,47 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.runtime.webmonitor.security; + +import org.apache.flink.util.Preconditions; + +/** Authenticated JobManager web user for a single HTTP request. */ +public final class JobManagerAuthenticatedUser { + + private final String userName; + private final String principal; + private final String type; + + JobManagerAuthenticatedUser(String userName, String principal, String type) { + this.userName = Preconditions.checkNotNull(userName); + this.principal = Preconditions.checkNotNull(principal); + this.type = Preconditions.checkNotNull(type); + } + + public String getUserName() { + return userName; + } + + public String getPrincipal() { + return principal; + } + + public String getType() { + return type; + } +} diff --git a/flink-runtime/src/main/java/org/apache/flink/runtime/webmonitor/security/JobManagerAuthenticatedUserHandler.java b/flink-runtime/src/main/java/org/apache/flink/runtime/webmonitor/security/JobManagerAuthenticatedUserHandler.java new file mode 100644 index 0000000000000..9a5afe97063a7 --- /dev/null +++ b/flink-runtime/src/main/java/org/apache/flink/runtime/webmonitor/security/JobManagerAuthenticatedUserHandler.java @@ -0,0 +1,109 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.runtime.webmonitor.security; + +import org.apache.flink.runtime.rest.handler.router.RoutedRequest; +import org.apache.flink.runtime.rest.handler.util.HandlerUtils; +import org.apache.flink.runtime.rest.messages.ResponseBody; + +import org.apache.flink.shaded.jackson2.com.fasterxml.jackson.annotation.JsonCreator; +import org.apache.flink.shaded.jackson2.com.fasterxml.jackson.annotation.JsonProperty; +import org.apache.flink.shaded.netty4.io.netty.channel.ChannelHandler; +import org.apache.flink.shaded.netty4.io.netty.channel.ChannelHandlerContext; +import org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpHeaderNames; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpResponseStatus; + +import javax.annotation.Nullable; + +import java.util.Collections; +import java.util.HashMap; +import java.util.Map; + +/** Handler for exposing the current JobManager web user to the dashboard. */ +@ChannelHandler.Sharable +public final class JobManagerAuthenticatedUserHandler + extends SimpleChannelInboundHandler> { + + private static final Map NO_STORE_HEADERS = createNoStoreHeaders(); + + @Override + protected void channelRead0(ChannelHandlerContext ctx, RoutedRequest routedRequest) { + JobManagerAuthenticatedUserResponseBody response = + JobManagerWebAuthenticationHandler.getAuthenticatedUser(ctx) + .map(JobManagerAuthenticatedUserResponseBody::authenticated) + .orElseGet(JobManagerAuthenticatedUserResponseBody::unauthenticated); + + HandlerUtils.sendResponse( + ctx, routedRequest.getRequest(), response, HttpResponseStatus.OK, NO_STORE_HEADERS); + } + + private static Map createNoStoreHeaders() { + Map headers = new HashMap<>(); + headers.put(HttpHeaderNames.CACHE_CONTROL.toString(), "no-store"); + headers.put(HttpHeaderNames.PRAGMA.toString(), "no-cache"); + return Collections.unmodifiableMap(headers); + } + + /** Response body for the current authenticated JobManager web user. */ + public static final class JobManagerAuthenticatedUserResponseBody implements ResponseBody { + + public static final String FIELD_NAME_AUTHENTICATED = "authenticated"; + public static final String FIELD_NAME_USER = "user"; + public static final String FIELD_NAME_PRINCIPAL = "principal"; + public static final String FIELD_NAME_TYPE = "type"; + + @JsonProperty(FIELD_NAME_AUTHENTICATED) + private final boolean authenticated; + + @JsonProperty(FIELD_NAME_USER) + @Nullable + private final String user; + + @JsonProperty(FIELD_NAME_PRINCIPAL) + @Nullable + private final String principal; + + @JsonProperty(FIELD_NAME_TYPE) + @Nullable + private final String type; + + @JsonCreator + public JobManagerAuthenticatedUserResponseBody( + @JsonProperty(FIELD_NAME_AUTHENTICATED) boolean authenticated, + @Nullable @JsonProperty(FIELD_NAME_USER) String user, + @Nullable @JsonProperty(FIELD_NAME_PRINCIPAL) String principal, + @Nullable @JsonProperty(FIELD_NAME_TYPE) String type) { + this.authenticated = authenticated; + this.user = user; + this.principal = principal; + this.type = type; + } + + static JobManagerAuthenticatedUserResponseBody authenticated( + JobManagerAuthenticatedUser user) { + return new JobManagerAuthenticatedUserResponseBody( + true, user.getUserName(), user.getPrincipal(), user.getType()); + } + + static JobManagerAuthenticatedUserResponseBody unauthenticated() { + return new JobManagerAuthenticatedUserResponseBody(false, null, null, null); + } + } +} diff --git a/flink-runtime/src/main/java/org/apache/flink/runtime/webmonitor/security/JobManagerAuthenticatedUserHeaders.java b/flink-runtime/src/main/java/org/apache/flink/runtime/webmonitor/security/JobManagerAuthenticatedUserHeaders.java new file mode 100644 index 0000000000000..b507d6010341f --- /dev/null +++ b/flink-runtime/src/main/java/org/apache/flink/runtime/webmonitor/security/JobManagerAuthenticatedUserHeaders.java @@ -0,0 +1,81 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.runtime.webmonitor.security; + +import org.apache.flink.runtime.rest.HttpMethodWrapper; +import org.apache.flink.runtime.rest.messages.EmptyMessageParameters; +import org.apache.flink.runtime.rest.messages.EmptyRequestBody; +import org.apache.flink.runtime.rest.messages.RuntimeMessageHeaders; + +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpResponseStatus; + +/** Message headers for exposing the current JobManager web user to the dashboard. */ +public final class JobManagerAuthenticatedUserHeaders + implements RuntimeMessageHeaders< + EmptyRequestBody, + JobManagerAuthenticatedUserHandler.JobManagerAuthenticatedUserResponseBody, + EmptyMessageParameters> { + + private static final JobManagerAuthenticatedUserHeaders INSTANCE = + new JobManagerAuthenticatedUserHeaders(); + + public static final String URL = "/auth/user"; + + private JobManagerAuthenticatedUserHeaders() {} + + @Override + public Class getRequestClass() { + return EmptyRequestBody.class; + } + + @Override + public HttpMethodWrapper getHttpMethod() { + return HttpMethodWrapper.GET; + } + + @Override + public String getTargetRestEndpointURL() { + return URL; + } + + @Override + public Class + getResponseClass() { + return JobManagerAuthenticatedUserHandler.JobManagerAuthenticatedUserResponseBody.class; + } + + @Override + public HttpResponseStatus getResponseStatusCode() { + return HttpResponseStatus.OK; + } + + @Override + public EmptyMessageParameters getUnresolvedMessageParameters() { + return EmptyMessageParameters.getInstance(); + } + + @Override + public String getDescription() { + return "Returns the current authenticated JobManager web user."; + } + + public static JobManagerAuthenticatedUserHeaders getInstance() { + return INSTANCE; + } +} diff --git a/flink-runtime/src/main/java/org/apache/flink/runtime/webmonitor/security/JobManagerAuthenticationTokenSigner.java b/flink-runtime/src/main/java/org/apache/flink/runtime/webmonitor/security/JobManagerAuthenticationTokenSigner.java new file mode 100644 index 0000000000000..6077baca74096 --- /dev/null +++ b/flink-runtime/src/main/java/org/apache/flink/runtime/webmonitor/security/JobManagerAuthenticationTokenSigner.java @@ -0,0 +1,126 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.runtime.webmonitor.security; + +import org.apache.hadoop.security.authentication.client.AuthenticationException; +import org.apache.hadoop.security.authentication.server.AuthenticationToken; + +import javax.crypto.Mac; +import javax.crypto.spec.SecretKeySpec; + +import java.nio.charset.StandardCharsets; +import java.security.InvalidKeyException; +import java.security.MessageDigest; +import java.security.NoSuchAlgorithmException; +import java.time.Clock; +import java.time.Duration; +import java.util.Base64; +import java.util.Optional; + +import static org.apache.flink.util.Preconditions.checkArgument; +import static org.apache.flink.util.Preconditions.checkNotNull; + +/** Signs and verifies Hadoop-compatible JobManager web authentication tokens. */ +final class JobManagerAuthenticationTokenSigner { + + static final String TOKEN_TYPE_KERBEROS = "kerberos"; + + private static final String SIGNATURE_SEPARATOR = "&s="; + private static final String SIGNING_ALGORITHM = "HmacSHA256"; + + private final byte[] secret; + private final Duration tokenValidity; + private final Clock clock; + + JobManagerAuthenticationTokenSigner(byte[] secret, Duration tokenValidity, Clock clock) { + this.secret = checkNotNull(secret, "secret must not be null").clone(); + checkArgument(this.secret.length > 0, "secret must not be empty"); + this.tokenValidity = checkNotNull(tokenValidity, "tokenValidity must not be null"); + this.clock = checkNotNull(clock, "clock must not be null"); + } + + String signToken(String userName, String principal) { + AuthenticationToken token = + new AuthenticationToken(userName, principal, TOKEN_TYPE_KERBEROS); + token.setExpires(clock.millis() + tokenValidity.toMillis()); + return sign(token.toString()); + } + + Optional verifyToken(String signedToken) { + if (signedToken == null || signedToken.isEmpty()) { + return Optional.empty(); + } + try { + AuthenticationToken token = + AuthenticationToken.parse(verifyAndExtract(stripQuotes(signedToken))); + if (!TOKEN_TYPE_KERBEROS.equals(token.getType())) { + return Optional.empty(); + } + if (token.getExpires() != -1 && clock.millis() > token.getExpires()) { + return Optional.empty(); + } + return Optional.of(token); + } catch (AuthenticationException | IllegalArgumentException e) { + return Optional.empty(); + } + } + + private String sign(String rawToken) { + return rawToken + SIGNATURE_SEPARATOR + computeSignature(rawToken); + } + + private String verifyAndExtract(String signedToken) { + int index = signedToken.lastIndexOf(SIGNATURE_SEPARATOR); + if (index < 0) { + throw new IllegalArgumentException("Authentication token is not signed."); + } + + String rawToken = signedToken.substring(0, index); + String expectedSignature = computeSignature(rawToken); + String actualSignature = signedToken.substring(index + SIGNATURE_SEPARATOR.length()); + if (!MessageDigest.isEqual( + expectedSignature.getBytes(StandardCharsets.UTF_8), + actualSignature.getBytes(StandardCharsets.UTF_8))) { + throw new IllegalArgumentException("Authentication token signature is invalid."); + } + return rawToken; + } + + private String computeSignature(String value) { + try { + Mac mac = Mac.getInstance(SIGNING_ALGORITHM); + mac.init(new SecretKeySpec(secret, SIGNING_ALGORITHM)); + return Base64.getEncoder() + .encodeToString(mac.doFinal(value.getBytes(StandardCharsets.UTF_8))); + } catch (NoSuchAlgorithmException | InvalidKeyException e) { + throw new IllegalStateException( + "Could not sign JobManager web authentication token.", e); + } + } + + private static String stripQuotes(String value) { + if (value != null + && value.length() >= 2 + && value.startsWith("\"") + && value.endsWith("\"")) { + return value.substring(1, value.length() - 1); + } + return value; + } +} diff --git a/flink-runtime/src/main/java/org/apache/flink/runtime/webmonitor/security/JobManagerSpnegoAuthenticationResult.java b/flink-runtime/src/main/java/org/apache/flink/runtime/webmonitor/security/JobManagerSpnegoAuthenticationResult.java new file mode 100644 index 0000000000000..e56f2a06a0d91 --- /dev/null +++ b/flink-runtime/src/main/java/org/apache/flink/runtime/webmonitor/security/JobManagerSpnegoAuthenticationResult.java @@ -0,0 +1,62 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.runtime.webmonitor.security; + +import org.apache.hadoop.security.authentication.server.AuthenticationToken; + +import javax.annotation.Nullable; + +import static org.apache.flink.util.Preconditions.checkNotNull; + +/** Result of one JobManager web SPNEGO authentication step. */ +final class JobManagerSpnegoAuthenticationResult { + + private final AuthenticationToken authenticationToken; + private final String authenticateHeader; + + private JobManagerSpnegoAuthenticationResult( + @Nullable AuthenticationToken authenticationToken, + @Nullable String authenticateHeader) { + this.authenticationToken = authenticationToken; + this.authenticateHeader = authenticateHeader; + } + + static JobManagerSpnegoAuthenticationResult challenge(String authenticateHeader) { + return new JobManagerSpnegoAuthenticationResult(null, checkNotNull(authenticateHeader)); + } + + static JobManagerSpnegoAuthenticationResult authenticated( + AuthenticationToken authenticationToken, @Nullable String authenticateHeader) { + return new JobManagerSpnegoAuthenticationResult( + checkNotNull(authenticationToken), authenticateHeader); + } + + boolean isAuthenticated() { + return authenticationToken != null; + } + + AuthenticationToken authenticationToken() { + return checkNotNull(authenticationToken); + } + + @Nullable + String authenticateHeader() { + return authenticateHeader; + } +} diff --git a/flink-runtime/src/main/java/org/apache/flink/runtime/webmonitor/security/JobManagerSpnegoAuthenticator.java b/flink-runtime/src/main/java/org/apache/flink/runtime/webmonitor/security/JobManagerSpnegoAuthenticator.java new file mode 100644 index 0000000000000..be58c232fd6f9 --- /dev/null +++ b/flink-runtime/src/main/java/org/apache/flink/runtime/webmonitor/security/JobManagerSpnegoAuthenticator.java @@ -0,0 +1,283 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.runtime.webmonitor.security; + +import org.apache.flink.util.ConfigurationException; + +import org.apache.hadoop.security.authentication.client.AuthenticationException; +import org.apache.hadoop.security.authentication.server.AuthenticationToken; +import org.apache.hadoop.security.authentication.util.KerberosName; +import org.apache.hadoop.security.authentication.util.KerberosUtil; +import org.ietf.jgss.GSSContext; +import org.ietf.jgss.GSSCredential; +import org.ietf.jgss.GSSException; +import org.ietf.jgss.GSSManager; +import org.ietf.jgss.Oid; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import javax.annotation.Nullable; +import javax.security.auth.Subject; +import javax.security.auth.kerberos.KerberosKey; +import javax.security.auth.kerberos.KerberosPrincipal; +import javax.security.auth.kerberos.KeyTab; + +import java.io.File; +import java.io.IOException; +import java.security.Principal; +import java.security.PrivilegedActionException; +import java.security.PrivilegedExceptionAction; +import java.util.Base64; +import java.util.regex.Pattern; + +import static org.apache.flink.util.Preconditions.checkNotNull; + +/** Kerberos/SPNEGO acceptor for the JobManager web endpoint. */ +final class JobManagerSpnegoAuthenticator implements SpnegoAuthenticator { + + private static final Logger LOG = LoggerFactory.getLogger(JobManagerSpnegoAuthenticator.class); + + static final String AUTHENTICATION_SCHEME_NEGOTIATE = "Negotiate"; + static final String AUTHENTICATE_HEADER_NEGOTIATE = "Negotiate"; + + private static final Pattern HTTP_PRINCIPAL_PATTERN = Pattern.compile("HTTP/.*"); + + private final Subject serverSubject; + private final GSSManager gssManager; + + private JobManagerSpnegoAuthenticator(Subject serverSubject, GSSManager gssManager) { + this.serverSubject = checkNotNull(serverSubject); + this.gssManager = checkNotNull(gssManager); + } + + static JobManagerSpnegoAuthenticator create(JobManagerWebAuthenticationConfig config) + throws ConfigurationException { + File keytabFile = config.getKerberosKeytab().toFile(); + if (!keytabFile.isFile()) { + throw new ConfigurationException( + "JobManager web authentication Kerberos keytab does not exist or is not a file: " + + config.getKerberosKeytab()); + } + + Subject serverSubject = new Subject(); + KeyTab keytab = KeyTab.getInstance(keytabFile); + serverSubject.getPrivateCredentials().add(keytab); + + if ("*".equals(config.getKerberosPrincipal())) { + addWildcardHttpPrincipals(serverSubject, config.getKerberosKeytab().toString()); + } else { + addPrincipal( + serverSubject, + keytab, + config.getKerberosPrincipal(), + config.getKerberosKeytab().toString()); + } + + configureKerberosNameRules(config); + + GSSManager gssManager; + try { + gssManager = + Subject.doAs( + serverSubject, + (PrivilegedExceptionAction) GSSManager::getInstance); + } catch (PrivilegedActionException e) { + throw new ConfigurationException( + "Could not initialize JobManager web SPNEGO acceptor.", e.getException()); + } + + return new JobManagerSpnegoAuthenticator(serverSubject, gssManager); + } + + @Override + public JobManagerSpnegoAuthenticationResult authenticate(String authorizationHeader) + throws AuthenticationException { + if (!hasNegotiateScheme(authorizationHeader)) { + return JobManagerSpnegoAuthenticationResult.challenge(AUTHENTICATE_HEADER_NEGOTIATE); + } + + String encodedClientToken = + authorizationHeader.substring(AUTHENTICATION_SCHEME_NEGOTIATE.length()).trim(); + if (encodedClientToken.isEmpty()) { + throw new AuthenticationException("Missing SPNEGO token."); + } + + byte[] clientToken; + try { + clientToken = Base64.getDecoder().decode(encodedClientToken); + } catch (IllegalArgumentException e) { + throw new AuthenticationException("Invalid SPNEGO token encoding.", e); + } + + try { + String serverPrincipal = KerberosUtil.getTokenServerName(clientToken); + if (!serverPrincipal.startsWith("HTTP/")) { + throw new AuthenticationException( + "Invalid SPNEGO server principal in client token."); + } + return Subject.doAs( + serverSubject, + (PrivilegedExceptionAction) + () -> runWithPrincipal(serverPrincipal, clientToken)); + } catch (PrivilegedActionException e) { + Throwable cause = e.getException(); + if (cause instanceof AuthenticationException) { + throw (AuthenticationException) cause; + } + throw new AuthenticationException(cause); + } catch (AuthenticationException e) { + throw e; + } catch (Exception e) { + throw new AuthenticationException(e); + } + } + + private JobManagerSpnegoAuthenticationResult runWithPrincipal( + String serverPrincipal, byte[] clientToken) throws GSSException, IOException { + GSSContext gssContext = null; + GSSCredential gssCredential = null; + try { + LOG.trace( + "Starting JobManager web SPNEGO step for server principal {}.", + serverPrincipal); + gssCredential = + gssManager.createCredential( + gssManager.createName( + serverPrincipal, KerberosUtil.NT_GSS_KRB5_PRINCIPAL_OID), + GSSCredential.INDEFINITE_LIFETIME, + new Oid[] { + KerberosUtil.GSS_SPNEGO_MECH_OID, KerberosUtil.GSS_KRB5_MECH_OID + }, + GSSCredential.ACCEPT_ONLY); + gssContext = gssManager.createContext(gssCredential); + byte[] serverToken = gssContext.acceptSecContext(clientToken, 0, clientToken.length); + String authenticateHeader = createAuthenticateHeader(serverToken); + + if (!gssContext.isEstablished()) { + LOG.trace("JobManager web SPNEGO step is not complete yet."); + return JobManagerSpnegoAuthenticationResult.challenge( + authenticateHeader == null + ? AUTHENTICATE_HEADER_NEGOTIATE + : authenticateHeader); + } + + String clientPrincipal = gssContext.getSrcName().toString(); + String userName = new KerberosName(clientPrincipal).getShortName(); + AuthenticationToken token = + new AuthenticationToken( + userName, + clientPrincipal, + JobManagerAuthenticationTokenSigner.TOKEN_TYPE_KERBEROS); + LOG.trace("JobManager web SPNEGO completed for client principal {}.", clientPrincipal); + return JobManagerSpnegoAuthenticationResult.authenticated(token, authenticateHeader); + } finally { + if (gssContext != null) { + gssContext.dispose(); + } + if (gssCredential != null) { + gssCredential.dispose(); + } + } + } + + @Nullable + private static String createAuthenticateHeader(byte[] serverToken) { + if (serverToken == null || serverToken.length == 0) { + return null; + } + return AUTHENTICATE_HEADER_NEGOTIATE + + " " + + Base64.getEncoder().encodeToString(serverToken); + } + + private static boolean hasNegotiateScheme(String authorizationHeader) { + return authorizationHeader != null + && authorizationHeader.length() >= AUTHENTICATION_SCHEME_NEGOTIATE.length() + && authorizationHeader.regionMatches( + true, + 0, + AUTHENTICATION_SCHEME_NEGOTIATE, + 0, + AUTHENTICATION_SCHEME_NEGOTIATE.length()) + && (authorizationHeader.length() == AUTHENTICATION_SCHEME_NEGOTIATE.length() + || Character.isWhitespace( + authorizationHeader.charAt( + AUTHENTICATION_SCHEME_NEGOTIATE.length()))); + } + + private static void addWildcardHttpPrincipals(Subject serverSubject, String keytab) + throws ConfigurationException { + String[] principals; + try { + principals = KerberosUtil.getPrincipalNames(keytab, HTTP_PRINCIPAL_PATTERN); + } catch (IOException e) { + throw new ConfigurationException( + "Could not read JobManager web authentication Kerberos keytab.", e); + } + if (principals.length == 0) { + throw new ConfigurationException( + "JobManager web authentication Kerberos keytab does not contain any HTTP principals."); + } + for (String principal : principals) { + addPrincipal(serverSubject, null, principal, keytab); + } + } + + private static void addPrincipal( + Subject serverSubject, KeyTab keytab, String principal, String keytabPath) + throws ConfigurationException { + try { + Principal kerberosPrincipal = new KerberosPrincipal(principal); + if (keytab != null) { + KerberosKey[] keys = keytab.getKeys((KerberosPrincipal) kerberosPrincipal); + if (keys.length == 0) { + throw new ConfigurationException( + String.format( + "JobManager web authentication Kerberos keytab '%s' does not contain principal '%s'.", + keytabPath, principal)); + } + for (KerberosKey key : keys) { + key.destroy(); + } + } + serverSubject.getPrincipals().add(kerberosPrincipal); + LOG.info( + "Using JobManager web SPNEGO keytab {} for principal {}.", + keytabPath, + principal); + } catch (IllegalArgumentException e) { + throw new ConfigurationException( + "Invalid JobManager web authentication Kerberos principal: " + principal, e); + } catch (javax.security.auth.DestroyFailedException e) { + throw new ConfigurationException( + "Could not validate JobManager web authentication Kerberos keytab.", e); + } + } + + private static void configureKerberosNameRules(JobManagerWebAuthenticationConfig config) + throws ConfigurationException { + try { + KerberosName.setRules(config.getKerberosNameRules()); + KerberosName.setRuleMechanism(KerberosName.DEFAULT_MECHANISM); + } catch (IllegalArgumentException e) { + throw new ConfigurationException( + "Invalid JobManager web SPNEGO Kerberos name rules.", e); + } + } +} diff --git a/flink-runtime/src/main/java/org/apache/flink/runtime/webmonitor/security/JobManagerWebAuthenticationConfig.java b/flink-runtime/src/main/java/org/apache/flink/runtime/webmonitor/security/JobManagerWebAuthenticationConfig.java new file mode 100644 index 0000000000000..a9350406403ee --- /dev/null +++ b/flink-runtime/src/main/java/org/apache/flink/runtime/webmonitor/security/JobManagerWebAuthenticationConfig.java @@ -0,0 +1,295 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.runtime.webmonitor.security; + +import org.apache.flink.configuration.Configuration; +import org.apache.flink.configuration.RestOptions; +import org.apache.flink.configuration.SecurityOptions; +import org.apache.flink.configuration.WebOptions.WebAuthenticationType; +import org.apache.flink.util.ConfigurationException; +import org.apache.flink.util.StringUtils; + +import org.apache.hadoop.security.authentication.util.KerberosUtil; + +import java.io.IOException; +import java.net.InetAddress; +import java.net.UnknownHostException; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.security.SecureRandom; +import java.time.Duration; +import java.util.Locale; +import java.util.Optional; + +import static org.apache.flink.configuration.WebOptions.AUTHENTICATION_COOKIE_PATH; +import static org.apache.flink.configuration.WebOptions.AUTHENTICATION_KERBEROS_KEYTAB; +import static org.apache.flink.configuration.WebOptions.AUTHENTICATION_KERBEROS_NAME_RULES; +import static org.apache.flink.configuration.WebOptions.AUTHENTICATION_KERBEROS_PRINCIPAL; +import static org.apache.flink.configuration.WebOptions.AUTHENTICATION_SIGNATURE_SECRET; +import static org.apache.flink.configuration.WebOptions.AUTHENTICATION_SIGNATURE_SECRET_FILE; +import static org.apache.flink.configuration.WebOptions.AUTHENTICATION_TOKEN_VALIDITY; +import static org.apache.flink.configuration.WebOptions.AUTHENTICATION_TYPE; + +/** Parsed and validated JobManager web authentication configuration. */ +final class JobManagerWebAuthenticationConfig { + + private static final int RANDOM_SECRET_BYTES = 32; + + private final String kerberosPrincipal; + private final Path kerberosKeytab; + private final String kerberosNameRules; + private final Duration tokenValidity; + private final String cookiePath; + private final byte[] signatureSecret; + private final boolean secureCookie; + + private JobManagerWebAuthenticationConfig( + String kerberosPrincipal, + Path kerberosKeytab, + String kerberosNameRules, + Duration tokenValidity, + String cookiePath, + byte[] signatureSecret, + boolean secureCookie) { + this.kerberosPrincipal = kerberosPrincipal; + this.kerberosKeytab = kerberosKeytab; + this.kerberosNameRules = kerberosNameRules; + this.tokenValidity = tokenValidity; + this.cookiePath = cookiePath; + this.signatureSecret = signatureSecret.clone(); + this.secureCookie = secureCookie; + } + + static Optional fromConfiguration( + Configuration configuration) throws ConfigurationException { + WebAuthenticationType type = getAuthenticationType(configuration); + if (type == WebAuthenticationType.NONE) { + return Optional.empty(); + } + if (type != WebAuthenticationType.KERBEROS) { + throw new ConfigurationException( + "Unsupported " + AUTHENTICATION_TYPE.key() + " value: " + type); + } + + return Optional.of( + new JobManagerWebAuthenticationConfig( + resolvePrincipal( + required(configuration, AUTHENTICATION_KERBEROS_PRINCIPAL), + configuration), + resolveKeytab(required(configuration, AUTHENTICATION_KERBEROS_KEYTAB)), + configuration.get(AUTHENTICATION_KERBEROS_NAME_RULES), + validateTokenValidity(configuration.get(AUTHENTICATION_TOKEN_VALIDITY)), + validateCookiePath(configuration.get(AUTHENTICATION_COOKIE_PATH)), + resolveSignatureSecret(configuration), + SecurityOptions.isRestSSLEnabled(configuration))); + } + + String getKerberosPrincipal() { + return kerberosPrincipal; + } + + Path getKerberosKeytab() { + return kerberosKeytab; + } + + String getKerberosNameRules() { + return kerberosNameRules; + } + + Duration getTokenValidity() { + return tokenValidity; + } + + String getCookiePath() { + return cookiePath; + } + + byte[] getSignatureSecret() { + return signatureSecret.clone(); + } + + boolean isSecureCookie() { + return secureCookie; + } + + private static WebAuthenticationType getAuthenticationType(Configuration configuration) + throws ConfigurationException { + try { + return configuration.get(AUTHENTICATION_TYPE); + } catch (IllegalArgumentException e) { + throw new ConfigurationException( + "Unsupported " + AUTHENTICATION_TYPE.key() + " value.", e); + } + } + + private static String resolvePrincipal(String principal, Configuration configuration) + throws ConfigurationException { + if ("*".equals(principal)) { + return principal; + } + String resolvedPrincipal = principal.replace("_HOST", resolveHostName(configuration)); + if (!resolvedPrincipal.startsWith("HTTP/")) { + throw new ConfigurationException( + AUTHENTICATION_KERBEROS_PRINCIPAL.key() + " must start with HTTP/ or be '*'."); + } + return resolvedPrincipal; + } + + private static String resolveHostName(Configuration configuration) + throws ConfigurationException { + Optional configuredHost = + firstUsableHost( + configuration.getOptional(RestOptions.ADDRESS), + configuration.getOptional(RestOptions.BIND_ADDRESS)); + if (configuredHost.isPresent()) { + try { + return InetAddress.getByName(configuredHost.get()) + .getCanonicalHostName() + .toLowerCase(Locale.ROOT); + } catch (UnknownHostException e) { + throw new ConfigurationException( + "Could not resolve _HOST in JobManager web Kerberos principal.", e); + } + } + try { + return KerberosUtil.getLocalHostName().toLowerCase(Locale.ROOT); + } catch (UnknownHostException e) { + throw new ConfigurationException( + "Could not resolve local hostname for JobManager web Kerberos principal.", e); + } + } + + @SafeVarargs + private static Optional firstUsableHost(Optional... hosts) { + for (Optional host : hosts) { + Optional usableHost = + host.map(String::trim) + .filter(value -> !value.isEmpty()) + .filter(value -> !isWildcardAddress(value)); + if (usableHost.isPresent()) { + return usableHost; + } + } + return Optional.empty(); + } + + private static boolean isWildcardAddress(String value) { + return "0.0.0.0".equals(value) || "::".equals(value) || "[::]".equals(value); + } + + private static Path resolveKeytab(String keytab) throws ConfigurationException { + Path keytabPath = Path.of(keytab); + if (!Files.isRegularFile(keytabPath) || !Files.isReadable(keytabPath)) { + throw new ConfigurationException( + AUTHENTICATION_KERBEROS_KEYTAB.key() + + " does not point to a readable regular file: " + + keytabPath); + } + return keytabPath; + } + + private static Duration validateTokenValidity(Duration tokenValidity) + throws ConfigurationException { + if (tokenValidity == null || tokenValidity.isZero() || tokenValidity.isNegative()) { + throw new ConfigurationException( + AUTHENTICATION_TOKEN_VALIDITY.key() + " must be greater than 0."); + } + return tokenValidity; + } + + private static String validateCookiePath(String cookiePath) throws ConfigurationException { + if (StringUtils.isNullOrWhitespaceOnly(cookiePath) || !cookiePath.startsWith("/")) { + throw new ConfigurationException( + AUTHENTICATION_COOKIE_PATH.key() + " must be a non-empty absolute path."); + } + return cookiePath; + } + + private static byte[] resolveSignatureSecret(Configuration configuration) + throws ConfigurationException { + Optional configuredSecret = + optionalNonBlank(configuration, AUTHENTICATION_SIGNATURE_SECRET); + Optional configuredSecretFile = + optionalNonBlank(configuration, AUTHENTICATION_SIGNATURE_SECRET_FILE); + + if (configuredSecret.isPresent() && configuredSecretFile.isPresent()) { + throw new ConfigurationException( + "Only one of " + + AUTHENTICATION_SIGNATURE_SECRET.key() + + " and " + + AUTHENTICATION_SIGNATURE_SECRET_FILE.key() + + " may be configured."); + } + + if (configuredSecret.isPresent()) { + return configuredSecret.get().getBytes(StandardCharsets.UTF_8); + } + if (configuredSecretFile.isPresent()) { + return readSecretFile(configuredSecretFile.get()); + } + return createRandomSecret(); + } + + private static byte[] readSecretFile(String secretFile) throws ConfigurationException { + try { + String secret = Files.readString(Path.of(secretFile), StandardCharsets.UTF_8).trim(); + if (secret.isEmpty()) { + throw new ConfigurationException( + AUTHENTICATION_SIGNATURE_SECRET_FILE.key() + " must not be empty."); + } + return secret.getBytes(StandardCharsets.UTF_8); + } catch (IOException e) { + throw new ConfigurationException( + "Could not read " + + AUTHENTICATION_SIGNATURE_SECRET_FILE.key() + + ": " + + secretFile, + e); + } + } + + private static byte[] createRandomSecret() { + byte[] secret = new byte[RANDOM_SECRET_BYTES]; + new SecureRandom().nextBytes(secret); + return secret; + } + + private static String required( + Configuration configuration, org.apache.flink.configuration.ConfigOption option) + throws ConfigurationException { + return optionalNonBlank(configuration, option) + .orElseThrow( + () -> + new ConfigurationException( + option.key() + + " must be configured when " + + AUTHENTICATION_TYPE.key() + + " is KERBEROS.")); + } + + private static Optional optionalNonBlank( + Configuration configuration, + org.apache.flink.configuration.ConfigOption option) { + return configuration + .getOptional(option) + .map(String::trim) + .filter(value -> !value.isEmpty()); + } +} diff --git a/flink-runtime/src/main/java/org/apache/flink/runtime/webmonitor/security/JobManagerWebAuthenticationHandler.java b/flink-runtime/src/main/java/org/apache/flink/runtime/webmonitor/security/JobManagerWebAuthenticationHandler.java new file mode 100644 index 0000000000000..e5c8760534bfa --- /dev/null +++ b/flink-runtime/src/main/java/org/apache/flink/runtime/webmonitor/security/JobManagerWebAuthenticationHandler.java @@ -0,0 +1,309 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.runtime.webmonitor.security; + +import org.apache.flink.configuration.Configuration; +import org.apache.flink.util.ConfigurationException; + +import org.apache.flink.shaded.netty4.io.netty.buffer.Unpooled; +import org.apache.flink.shaded.netty4.io.netty.channel.ChannelDuplexHandler; +import org.apache.flink.shaded.netty4.io.netty.channel.ChannelFutureListener; +import org.apache.flink.shaded.netty4.io.netty.channel.ChannelHandler; +import org.apache.flink.shaded.netty4.io.netty.channel.ChannelHandlerContext; +import org.apache.flink.shaded.netty4.io.netty.channel.ChannelPromise; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.DefaultFullHttpResponse; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.FullHttpRequest; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.FullHttpResponse; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpHeaderNames; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpHeaderValues; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpHeaders; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpResponse; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpResponseStatus; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpUtil; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpVersion; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.cookie.Cookie; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.cookie.ServerCookieDecoder; +import org.apache.flink.shaded.netty4.io.netty.util.AttributeKey; +import org.apache.flink.shaded.netty4.io.netty.util.ReferenceCountUtil; + +import org.apache.hadoop.security.authentication.client.AuthenticatedURL; +import org.apache.hadoop.security.authentication.client.AuthenticationException; +import org.apache.hadoop.security.authentication.server.AuthenticationToken; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import javax.annotation.Nullable; + +import java.time.Clock; +import java.util.Map; +import java.util.Optional; +import java.util.Set; + +import static org.apache.flink.util.Preconditions.checkNotNull; + +/** Netty handler that enforces JobManager web SPNEGO authentication. */ +public final class JobManagerWebAuthenticationHandler extends ChannelDuplexHandler { + + private static final Logger LOG = + LoggerFactory.getLogger(JobManagerWebAuthenticationHandler.class); + + private static final AttributeKey RESPONSE_AUTHENTICATION_ATTRIBUTE = + AttributeKey.valueOf("jobmanager-web-spnego-response-authentication"); + private static final AttributeKey AUTHENTICATED_USER = + AttributeKey.valueOf("jobmanager-web-authenticated-user"); + + private final SpnegoAuthenticator authenticator; + private final JobManagerAuthenticationTokenSigner tokenSigner; + private final String cookiePath; + private final boolean secureCookie; + private final Map responseHeaders; + + JobManagerWebAuthenticationHandler( + SpnegoAuthenticator authenticator, + JobManagerAuthenticationTokenSigner tokenSigner, + String cookiePath, + boolean secureCookie, + Map responseHeaders) { + this.authenticator = checkNotNull(authenticator); + this.tokenSigner = checkNotNull(tokenSigner); + this.cookiePath = checkNotNull(cookiePath); + this.secureCookie = secureCookie; + this.responseHeaders = checkNotNull(responseHeaders); + } + + public static Optional createFactory( + Configuration configuration, Map responseHeaders) + throws ConfigurationException { + Optional config = + JobManagerWebAuthenticationConfig.fromConfiguration(configuration); + if (!config.isPresent()) { + return Optional.empty(); + } + + JobManagerWebAuthenticationConfig authenticationConfig = config.get(); + SpnegoAuthenticator authenticator = + JobManagerSpnegoAuthenticator.create(authenticationConfig); + JobManagerAuthenticationTokenSigner signer = + new JobManagerAuthenticationTokenSigner( + authenticationConfig.getSignatureSecret(), + authenticationConfig.getTokenValidity(), + Clock.systemUTC()); + + return Optional.of( + () -> + new JobManagerWebAuthenticationHandler( + authenticator, + signer, + authenticationConfig.getCookiePath(), + authenticationConfig.isSecureCookie(), + responseHeaders)); + } + + @Override + public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception { + if (!(msg instanceof FullHttpRequest)) { + ctx.fireChannelRead(msg); + return; + } + + FullHttpRequest request = (FullHttpRequest) msg; + Optional cookieToken = getAuthenticationTokenFromCookie(request); + if (cookieToken.isPresent()) { + setAuthenticatedUser(ctx, cookieToken.get()); + try { + ctx.fireChannelRead(msg); + } finally { + clearAuthenticatedUser(ctx); + } + return; + } + + try { + JobManagerSpnegoAuthenticationResult result = + authenticator.authenticate( + request.headers().get(HttpHeaderNames.AUTHORIZATION)); + if (!result.isAuthenticated()) { + sendResponse( + ctx, + request, + HttpResponseStatus.UNAUTHORIZED, + result.authenticateHeader(), + true); + return; + } + + AuthenticationToken authenticationToken = result.authenticationToken(); + String signedToken = + tokenSigner.signToken( + authenticationToken.getUserName(), authenticationToken.getName()); + ctx.channel() + .attr(RESPONSE_AUTHENTICATION_ATTRIBUTE) + .set(new ResponseAuthentication(signedToken, result.authenticateHeader())); + setAuthenticatedUser(ctx, authenticationToken); + try { + ctx.fireChannelRead(msg); + } finally { + clearAuthenticatedUser(ctx); + } + } catch (AuthenticationException e) { + LOG.debug("JobManager web SPNEGO authentication failed.", e); + clearAuthenticatedUser(ctx); + sendResponse(ctx, request, HttpResponseStatus.FORBIDDEN, null, true); + } + } + + @Override + public void write(ChannelHandlerContext ctx, Object msg, ChannelPromise promise) + throws Exception { + if (msg instanceof HttpResponse) { + ResponseAuthentication responseAuthentication = + ctx.channel().attr(RESPONSE_AUTHENTICATION_ATTRIBUTE).getAndSet(null); + if (responseAuthentication != null) { + HttpHeaders headers = ((HttpResponse) msg).headers(); + headers.add(HttpHeaderNames.SET_COOKIE, createAuthCookie(responseAuthentication)); + if (responseAuthentication.authenticateHeader() != null) { + headers.set( + HttpHeaderNames.WWW_AUTHENTICATE, + responseAuthentication.authenticateHeader()); + } + } + } + ctx.write(msg, promise); + } + + private Optional getAuthenticationTokenFromCookie( + FullHttpRequest request) { + for (String cookieHeader : request.headers().getAll(HttpHeaderNames.COOKIE)) { + try { + Set cookies = ServerCookieDecoder.STRICT.decode(cookieHeader); + for (Cookie cookie : cookies) { + if (AuthenticatedURL.AUTH_COOKIE.equals(cookie.name())) { + Optional token = + tokenSigner.verifyToken(cookie.value()); + if (token.isPresent()) { + return token; + } + } + } + } catch (IllegalArgumentException e) { + LOG.debug("Ignoring malformed JobManager web Cookie header.", e); + } + } + return Optional.empty(); + } + + private static void setAuthenticatedUser( + ChannelHandlerContext ctx, AuthenticationToken authenticationToken) { + ctx.channel() + .attr(AUTHENTICATED_USER) + .set( + new JobManagerAuthenticatedUser( + authenticationToken.getUserName(), + authenticationToken.getName(), + authenticationToken.getType())); + } + + private static void clearAuthenticatedUser(ChannelHandlerContext ctx) { + ctx.channel().attr(AUTHENTICATED_USER).set(null); + } + + public static Optional getAuthenticatedUser( + ChannelHandlerContext ctx) { + return Optional.ofNullable(ctx.channel().attr(AUTHENTICATED_USER).get()); + } + + private void sendResponse( + ChannelHandlerContext ctx, + FullHttpRequest request, + HttpResponseStatus status, + @Nullable String authenticateHeader, + boolean expireCookie) { + clearAuthenticatedUser(ctx); + FullHttpResponse response = + new DefaultFullHttpResponse(HttpVersion.HTTP_1_1, status, Unpooled.EMPTY_BUFFER); + responseHeaders.forEach((name, value) -> response.headers().set(name, value)); + response.headers().set(HttpHeaderNames.CONTENT_LENGTH, 0); + if (authenticateHeader != null) { + response.headers().set(HttpHeaderNames.WWW_AUTHENTICATE, authenticateHeader); + } + if (expireCookie) { + response.headers().add(HttpHeaderNames.SET_COOKIE, createExpiredAuthCookie()); + } + + boolean keepAlive = HttpUtil.isKeepAlive(request); + ReferenceCountUtil.release(request); + if (keepAlive) { + response.headers().set(HttpHeaderNames.CONNECTION, HttpHeaderValues.KEEP_ALIVE); + ctx.writeAndFlush(response); + } else { + ctx.writeAndFlush(response).addListener(ChannelFutureListener.CLOSE); + } + } + + private String createAuthCookie(ResponseAuthentication responseAuthentication) { + StringBuilder cookie = + new StringBuilder(AuthenticatedURL.AUTH_COOKIE) + .append("=\"") + .append(responseAuthentication.signedToken()) + .append("\"; Path=") + .append(cookiePath); + if (secureCookie) { + cookie.append("; Secure"); + } + cookie.append("; HttpOnly"); + return cookie.toString(); + } + + private String createExpiredAuthCookie() { + StringBuilder cookie = + new StringBuilder(AuthenticatedURL.AUTH_COOKIE) + .append("=; Path=") + .append(cookiePath) + .append("; Max-Age=0"); + if (secureCookie) { + cookie.append("; Secure"); + } + cookie.append("; HttpOnly"); + return cookie.toString(); + } + + /** Factory for per-channel JobManager web authentication handlers. */ + public interface Factory { + ChannelHandler createHandler(); + } + + private static final class ResponseAuthentication { + private final String signedToken; + @Nullable private final String authenticateHeader; + + private ResponseAuthentication(String signedToken, @Nullable String authenticateHeader) { + this.signedToken = signedToken; + this.authenticateHeader = authenticateHeader; + } + + private String signedToken() { + return signedToken; + } + + @Nullable + private String authenticateHeader() { + return authenticateHeader; + } + } +} diff --git a/flink-runtime/src/main/java/org/apache/flink/runtime/webmonitor/security/SpnegoAuthenticator.java b/flink-runtime/src/main/java/org/apache/flink/runtime/webmonitor/security/SpnegoAuthenticator.java new file mode 100644 index 0000000000000..c33d934a19596 --- /dev/null +++ b/flink-runtime/src/main/java/org/apache/flink/runtime/webmonitor/security/SpnegoAuthenticator.java @@ -0,0 +1,28 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.runtime.webmonitor.security; + +import org.apache.hadoop.security.authentication.client.AuthenticationException; + +/** Authentication step contract used by the JobManager web SPNEGO handler. */ +interface SpnegoAuthenticator { + + JobManagerSpnegoAuthenticationResult authenticate(String authorizationHeader) + throws AuthenticationException; +} diff --git a/flink-runtime/src/main/resources/META-INF/NOTICE b/flink-runtime/src/main/resources/META-INF/NOTICE index c453ff63d3563..8cb5803d591e5 100644 --- a/flink-runtime/src/main/resources/META-INF/NOTICE +++ b/flink-runtime/src/main/resources/META-INF/NOTICE @@ -7,3 +7,15 @@ The Apache Software Foundation (http://www.apache.org/). This project bundles the following dependencies under the Apache Software License 2.0. (http://www.apache.org/licenses/LICENSE-2.0.txt) - io.airlift:aircompressor:0.27 +- org.apache.hadoop:hadoop-auth +- commons-codec:commons-codec +- org.apache.httpcomponents:httpclient +- org.apache.httpcomponents:httpcore +- org.apache.hadoop.thirdparty:hadoop-shaded-guava +- org.apache.kerby:kerb-core +- org.apache.kerby:kerby-pkix +- org.apache.kerby:kerby-asn1 +- org.apache.kerby:kerby-util +- org.apache.kerby:kerb-util +- org.apache.kerby:kerby-config +- org.apache.kerby:kerb-crypto diff --git a/flink-runtime/src/test/java/org/apache/flink/runtime/rest/util/TestRestServerEndpoint.java b/flink-runtime/src/test/java/org/apache/flink/runtime/rest/util/TestRestServerEndpoint.java index c9b25fe17a57a..f44222b33aa78 100644 --- a/flink-runtime/src/test/java/org/apache/flink/runtime/rest/util/TestRestServerEndpoint.java +++ b/flink-runtime/src/test/java/org/apache/flink/runtime/rest/util/TestRestServerEndpoint.java @@ -25,12 +25,16 @@ import org.apache.flink.runtime.rest.handler.RestHandlerSpecification; import org.apache.flink.util.ConfigurationException; +import org.apache.flink.shaded.netty4.io.netty.channel.ChannelHandler; import org.apache.flink.shaded.netty4.io.netty.channel.ChannelInboundHandler; import java.io.IOException; import java.util.ArrayList; +import java.util.Collection; import java.util.List; import java.util.concurrent.CompletableFuture; +import java.util.function.Supplier; +import java.util.stream.Collectors; /** Utility {@link RestServerEndpoint} for setting up a rest server with a given set of handlers. */ public class TestRestServerEndpoint extends RestServerEndpoint { @@ -47,6 +51,8 @@ public static class Builder { private final Configuration configuration; private final List> handlers = new ArrayList<>(); + private final List> endpointSpecificChannelHandlerFactories = + new ArrayList<>(); private Builder(Configuration configuration) { this.configuration = configuration; @@ -63,8 +69,20 @@ public Builder withHandler(AbstractRestHandler handler) { return this; } + public Builder withEndpointSpecificChannelHandler(ChannelHandler channelHandler) { + this.endpointSpecificChannelHandlerFactories.add(() -> channelHandler); + return this; + } + + public Builder withEndpointSpecificChannelHandlerFactory( + Supplier channelHandlerFactory) { + this.endpointSpecificChannelHandlerFactories.add(channelHandlerFactory); + return this; + } + public TestRestServerEndpoint build() throws IOException, ConfigurationException { - return new TestRestServerEndpoint(configuration, handlers); + return new TestRestServerEndpoint( + configuration, handlers, endpointSpecificChannelHandlerFactories); } public TestRestServerEndpoint buildAndStart() throws Exception { @@ -76,13 +94,16 @@ public TestRestServerEndpoint buildAndStart() throws Exception { } private final List> handlers; + private final List> endpointSpecificChannelHandlerFactories; private TestRestServerEndpoint( final Configuration configuration, - final List> handlers) + final List> handlers, + final List> endpointSpecificChannelHandlerFactories) throws IOException, ConfigurationException { super(configuration); this.handlers = handlers; + this.endpointSpecificChannelHandlerFactories = endpointSpecificChannelHandlerFactories; } @Override @@ -91,6 +112,13 @@ protected List> initiali return this.handlers; } + @Override + protected Collection createEndpointSpecificChannelHandlers() { + return endpointSpecificChannelHandlerFactories.stream() + .map(Supplier::get) + .collect(Collectors.toList()); + } + @Override protected void startInternal() {} } diff --git a/flink-runtime/src/test/java/org/apache/flink/runtime/webmonitor/WebMonitorEndpointTest.java b/flink-runtime/src/test/java/org/apache/flink/runtime/webmonitor/WebMonitorEndpointTest.java index 3ad06659d8264..ab94f68d39b1a 100644 --- a/flink-runtime/src/test/java/org/apache/flink/runtime/webmonitor/WebMonitorEndpointTest.java +++ b/flink-runtime/src/test/java/org/apache/flink/runtime/webmonitor/WebMonitorEndpointTest.java @@ -27,10 +27,15 @@ import org.apache.flink.runtime.rest.handler.RestHandlerConfiguration; import org.apache.flink.runtime.rest.handler.legacy.metrics.VoidMetricFetcher; import org.apache.flink.runtime.util.TestingFatalErrorHandler; +import org.apache.flink.runtime.webmonitor.security.JobManagerAuthenticatedUserHeaders; +import org.apache.flink.util.ConfigurationException; import org.apache.flink.util.ExecutorUtils; import org.junit.jupiter.api.Test; +import java.net.HttpURLConnection; +import java.net.URI; +import java.nio.charset.StandardCharsets; import java.time.Duration; import java.util.UUID; import java.util.concurrent.CompletableFuture; @@ -38,6 +43,9 @@ import java.util.concurrent.ScheduledExecutorService; import java.util.concurrent.TimeUnit; +import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.assertThatThrownBy; + /** Tests for the {@link WebMonitorEndpoint}. */ class WebMonitorEndpointTest { @@ -45,6 +53,7 @@ class WebMonitorEndpointTest { void cleansUpExpiredExecutionGraphs() throws Exception { final Configuration configuration = new Configuration(); configuration.set(RestOptions.ADDRESS, "localhost"); + configuration.set(RestOptions.BIND_PORT, "0"); configuration.set(WebOptions.REFRESH_INTERVAL, Duration.ofMillis(5L)); final ScheduledExecutorService executor = Executors.newScheduledThreadPool(1); final long timeout = 10000L; @@ -75,4 +84,83 @@ void cleansUpExpiredExecutionGraphs() throws Exception { ExecutorUtils.gracefulShutdown(timeout, TimeUnit.MILLISECONDS, executor); } } + + @Test + void exposesAuthenticatedUserEndpointWhenWebAuthenticationIsDisabled() throws Exception { + final Configuration configuration = new Configuration(); + configuration.set(RestOptions.ADDRESS, "localhost"); + configuration.set(RestOptions.BIND_PORT, "0"); + final ScheduledExecutorService executor = Executors.newScheduledThreadPool(1); + final long timeout = 10000L; + + try (final WebMonitorEndpoint webMonitorEndpoint = + new WebMonitorEndpoint<>( + CompletableFuture::new, + configuration, + RestHandlerConfiguration.fromConfiguration(configuration), + CompletableFuture::new, + NoOpTransientBlobService.INSTANCE, + executor, + VoidMetricFetcher.INSTANCE, + new StandaloneLeaderElection(UUID.randomUUID()), + TestingExecutionGraphCache.newBuilder().build(), + new TestingFatalErrorHandler())) { + + webMonitorEndpoint.start(); + + HttpURLConnection connection = + (HttpURLConnection) + new URI( + "http", + null, + "localhost", + webMonitorEndpoint.getServerAddress().getPort(), + JobManagerAuthenticatedUserHeaders.URL, + null, + null) + .toURL() + .openConnection(); + connection.setRequestMethod("GET"); + + assertThat(connection.getResponseCode()).isEqualTo(200); + assertThat( + new String( + connection.getInputStream().readAllBytes(), + StandardCharsets.UTF_8)) + .contains("\"authenticated\":false"); + } finally { + ExecutorUtils.gracefulShutdown(timeout, TimeUnit.MILLISECONDS, executor); + } + } + + @Test + void failsFastForInvalidKerberosWebAuthenticationConfiguration() throws Exception { + final Configuration configuration = new Configuration(); + configuration.set(RestOptions.ADDRESS, "localhost"); + configuration.set( + WebOptions.AUTHENTICATION_TYPE, WebOptions.WebAuthenticationType.KERBEROS); + final ScheduledExecutorService executor = Executors.newScheduledThreadPool(1); + final long timeout = 10000L; + + try { + assertThatThrownBy( + () -> + new WebMonitorEndpoint<>( + CompletableFuture::new, + configuration, + RestHandlerConfiguration.fromConfiguration( + configuration), + CompletableFuture::new, + NoOpTransientBlobService.INSTANCE, + executor, + VoidMetricFetcher.INSTANCE, + new StandaloneLeaderElection(UUID.randomUUID()), + TestingExecutionGraphCache.newBuilder().build(), + new TestingFatalErrorHandler())) + .isInstanceOf(ConfigurationException.class) + .hasMessageContaining(WebOptions.AUTHENTICATION_KERBEROS_PRINCIPAL.key()); + } finally { + ExecutorUtils.gracefulShutdown(timeout, TimeUnit.MILLISECONDS, executor); + } + } } diff --git a/flink-runtime/src/test/java/org/apache/flink/runtime/webmonitor/security/JobManagerAuthenticatedUserHandlerTest.java b/flink-runtime/src/test/java/org/apache/flink/runtime/webmonitor/security/JobManagerAuthenticatedUserHandlerTest.java new file mode 100644 index 0000000000000..b2151a6a9cc8e --- /dev/null +++ b/flink-runtime/src/test/java/org/apache/flink/runtime/webmonitor/security/JobManagerAuthenticatedUserHandlerTest.java @@ -0,0 +1,162 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.runtime.webmonitor.security; + +import org.apache.flink.runtime.rest.handler.router.RouteResult; +import org.apache.flink.runtime.rest.handler.router.RoutedRequest; + +import org.apache.flink.shaded.netty4.io.netty.buffer.ByteBuf; +import org.apache.flink.shaded.netty4.io.netty.channel.ChannelHandlerContext; +import org.apache.flink.shaded.netty4.io.netty.channel.ChannelInboundHandlerAdapter; +import org.apache.flink.shaded.netty4.io.netty.channel.embedded.EmbeddedChannel; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.DefaultFullHttpRequest; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.FullHttpRequest; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpHeaderNames; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpMethod; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpResponse; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpResponseStatus; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpVersion; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.LastHttpContent; +import org.apache.flink.shaded.netty4.io.netty.util.ReferenceCountUtil; + +import org.apache.hadoop.security.authentication.client.AuthenticatedURL; +import org.junit.jupiter.api.Test; + +import java.nio.charset.StandardCharsets; +import java.time.Clock; +import java.time.Duration; +import java.time.Instant; +import java.time.ZoneOffset; +import java.util.Collections; + +import static org.assertj.core.api.Assertions.assertThat; + +/** Tests for {@link JobManagerAuthenticatedUserHandler}. */ +class JobManagerAuthenticatedUserHandlerTest { + + private static final Clock NOW = Clock.fixed(Instant.ofEpochMilli(10_000L), ZoneOffset.UTC); + private static final byte[] SECRET = "secret".getBytes(StandardCharsets.UTF_8); + + @Test + void shouldReturnUnauthenticatedResponseWhenAuthenticationIsDisabled() { + EmbeddedChannel channel = new EmbeddedChannel(new JobManagerAuthenticatedUserHandler()); + DefaultFullHttpRequest request = + new DefaultFullHttpRequest( + HttpVersion.HTTP_1_1, + HttpMethod.GET, + JobManagerAuthenticatedUserHeaders.URL); + + try { + assertThat(channel.writeInbound(routedRequest(request))).isFalse(); + + HttpResponse response = channel.readOutbound(); + assertThat(response.status()).isEqualTo(HttpResponseStatus.OK); + assertThat(response.headers().get(HttpHeaderNames.CACHE_CONTROL)).isEqualTo("no-store"); + ByteBuf content = channel.readOutbound(); + LastHttpContent lastContent = channel.readOutbound(); + try { + assertThat(content.toString(StandardCharsets.UTF_8)) + .contains("\"authenticated\":false"); + assertThat(lastContent).isSameAs(LastHttpContent.EMPTY_LAST_CONTENT); + } finally { + ReferenceCountUtil.release(content); + ReferenceCountUtil.release(lastContent); + } + } finally { + channel.finishAndReleaseAll(); + } + } + + @Test + void shouldReturnAuthenticatedUserFromCurrentRequest() { + JobManagerAuthenticationTokenSigner signer = + new JobManagerAuthenticationTokenSigner(SECRET, Duration.ofHours(1), NOW); + JobManagerAuthenticatedUserHandler authenticatedUserHandler = + new JobManagerAuthenticatedUserHandler(); + EmbeddedChannel channel = + new EmbeddedChannel( + new JobManagerWebAuthenticationHandler( + authorization -> + JobManagerSpnegoAuthenticationResult.challenge("Negotiate"), + signer, + "/", + false, + Collections.emptyMap()), + new ChannelInboundHandlerAdapter() { + @Override + public void channelRead(ChannelHandlerContext ctx, Object msg) + throws Exception { + if (msg instanceof FullHttpRequest) { + authenticatedUserHandler.channelRead( + ctx, routedRequest((FullHttpRequest) msg)); + } else { + super.channelRead(ctx, msg); + } + } + }); + DefaultFullHttpRequest request = + new DefaultFullHttpRequest( + HttpVersion.HTTP_1_1, + HttpMethod.GET, + JobManagerAuthenticatedUserHeaders.URL); + request.headers() + .set( + HttpHeaderNames.COOKIE, + AuthenticatedURL.AUTH_COOKIE + + "=\"" + + signer.signToken("alice", "alice@EXAMPLE.COM") + + "\""); + + try { + assertThat(channel.writeInbound(request)).isFalse(); + + HttpResponse response = channel.readOutbound(); + assertThat(response.status()).isEqualTo(HttpResponseStatus.OK); + ByteBuf content = channel.readOutbound(); + LastHttpContent lastContent = channel.readOutbound(); + try { + assertThat(content.toString(StandardCharsets.UTF_8)) + .contains("\"authenticated\":true") + .contains("\"user\":\"alice\"") + .contains("\"principal\":\"alice@EXAMPLE.COM\"") + .contains( + "\"type\":\"" + + JobManagerAuthenticationTokenSigner.TOKEN_TYPE_KERBEROS + + "\""); + assertThat(lastContent).isSameAs(LastHttpContent.EMPTY_LAST_CONTENT); + } finally { + ReferenceCountUtil.release(content); + ReferenceCountUtil.release(lastContent); + } + } finally { + channel.finishAndReleaseAll(); + } + } + + private static RoutedRequest routedRequest(FullHttpRequest request) { + return new RoutedRequest<>( + new RouteResult<>( + request.uri(), + request.uri(), + Collections.emptyMap(), + Collections.emptyMap(), + new JobManagerAuthenticatedUserHandler()), + request); + } +} diff --git a/flink-runtime/src/test/java/org/apache/flink/runtime/webmonitor/security/JobManagerAuthenticationTokenSignerTest.java b/flink-runtime/src/test/java/org/apache/flink/runtime/webmonitor/security/JobManagerAuthenticationTokenSignerTest.java new file mode 100644 index 0000000000000..2b2ffe518e261 --- /dev/null +++ b/flink-runtime/src/test/java/org/apache/flink/runtime/webmonitor/security/JobManagerAuthenticationTokenSignerTest.java @@ -0,0 +1,88 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.runtime.webmonitor.security; + +import org.apache.hadoop.security.authentication.server.AuthenticationToken; +import org.junit.jupiter.api.Test; + +import java.nio.charset.StandardCharsets; +import java.time.Clock; +import java.time.Duration; +import java.time.Instant; +import java.time.ZoneOffset; +import java.util.Optional; + +import static org.assertj.core.api.Assertions.assertThat; + +/** Tests for {@link JobManagerAuthenticationTokenSigner}. */ +class JobManagerAuthenticationTokenSignerTest { + + private static final byte[] SECRET = "shared-secret".getBytes(StandardCharsets.UTF_8); + private static final Clock NOW = Clock.fixed(Instant.ofEpochMilli(10_000L), ZoneOffset.UTC); + + @Test + void shouldSignAndVerifyAuthenticationToken() { + JobManagerAuthenticationTokenSigner signer = signer(NOW, Duration.ofHours(1)); + + Optional token = + signer.verifyToken(signer.signToken("alice", "alice@EXAMPLE.COM")); + + assertThat(token) + .hasValueSatisfying( + authenticationToken -> { + assertThat(authenticationToken.getUserName()).isEqualTo("alice"); + assertThat(authenticationToken.getName()) + .isEqualTo("alice@EXAMPLE.COM"); + assertThat(authenticationToken.getType()) + .isEqualTo( + JobManagerAuthenticationTokenSigner + .TOKEN_TYPE_KERBEROS); + }); + } + + @Test + void shouldRejectInvalidSignature() { + JobManagerAuthenticationTokenSigner signer = signer(NOW, Duration.ofHours(1)); + String signedToken = signer.signToken("alice", "alice@EXAMPLE.COM"); + + assertThat(signer.verifyToken(signedToken + "tampered")).isEmpty(); + } + + @Test + void shouldRejectExpiredToken() { + JobManagerAuthenticationTokenSigner signer = signer(NOW, Duration.ofMillis(1)); + String signedToken = signer.signToken("alice", "alice@EXAMPLE.COM"); + JobManagerAuthenticationTokenSigner verifier = + signer(Clock.offset(NOW, Duration.ofMillis(2)), Duration.ofMillis(1)); + + assertThat(verifier.verifyToken(signedToken)).isEmpty(); + } + + @Test + void shouldAcceptQuotedCookieValue() { + JobManagerAuthenticationTokenSigner signer = signer(NOW, Duration.ofHours(1)); + String signedToken = signer.signToken("alice", "alice@EXAMPLE.COM"); + + assertThat(signer.verifyToken("\"" + signedToken + "\"")).isPresent(); + } + + private static JobManagerAuthenticationTokenSigner signer(Clock clock, Duration tokenValidity) { + return new JobManagerAuthenticationTokenSigner(SECRET, tokenValidity, clock); + } +} diff --git a/flink-runtime/src/test/java/org/apache/flink/runtime/webmonitor/security/JobManagerWebAuthenticationConfigTest.java b/flink-runtime/src/test/java/org/apache/flink/runtime/webmonitor/security/JobManagerWebAuthenticationConfigTest.java new file mode 100644 index 0000000000000..3996ed8693c3e --- /dev/null +++ b/flink-runtime/src/test/java/org/apache/flink/runtime/webmonitor/security/JobManagerWebAuthenticationConfigTest.java @@ -0,0 +1,226 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.runtime.webmonitor.security; + +import org.apache.flink.configuration.Configuration; +import org.apache.flink.configuration.RestOptions; +import org.apache.flink.configuration.WebOptions; +import org.apache.flink.configuration.WebOptions.WebAuthenticationType; +import org.apache.flink.util.ConfigurationException; + +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.io.TempDir; + +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.time.Duration; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.assertThatThrownBy; + +/** Tests for {@link JobManagerWebAuthenticationConfig}. */ +class JobManagerWebAuthenticationConfigTest { + + @TempDir private Path tempDir; + + @Test + void shouldReturnEmptyConfigWhenAuthenticationIsDisabled() throws Exception { + assertThat(JobManagerWebAuthenticationConfig.fromConfiguration(new Configuration())) + .isEmpty(); + } + + @Test + void shouldRejectInvalidAuthenticationType() { + Configuration configuration = new Configuration(); + configuration.setString(WebOptions.AUTHENTICATION_TYPE.key(), "BASIC"); + + assertThatThrownBy(() -> JobManagerWebAuthenticationConfig.fromConfiguration(configuration)) + .isInstanceOf(ConfigurationException.class) + .hasMessageContaining(WebOptions.AUTHENTICATION_TYPE.key()); + } + + @Test + void shouldRequirePrincipalWhenKerberosIsEnabled() throws Exception { + Configuration configuration = kerberosConfiguration(); + configuration.set(WebOptions.AUTHENTICATION_KERBEROS_KEYTAB, createKeytab().toString()); + + assertThatThrownBy(() -> JobManagerWebAuthenticationConfig.fromConfiguration(configuration)) + .isInstanceOf(ConfigurationException.class) + .hasMessageContaining(WebOptions.AUTHENTICATION_KERBEROS_PRINCIPAL.key()); + } + + @Test + void shouldRequireKeytabWhenKerberosIsEnabled() { + Configuration configuration = kerberosConfiguration(); + configuration.set( + WebOptions.AUTHENTICATION_KERBEROS_PRINCIPAL, "HTTP/localhost@EXAMPLE.COM"); + + assertThatThrownBy(() -> JobManagerWebAuthenticationConfig.fromConfiguration(configuration)) + .isInstanceOf(ConfigurationException.class) + .hasMessageContaining(WebOptions.AUTHENTICATION_KERBEROS_KEYTAB.key()); + } + + @Test + void shouldResolveHostPlaceholderInPrincipalFromRestAddress() throws Exception { + Configuration configuration = validKerberosConfiguration(); + configuration.set(WebOptions.AUTHENTICATION_KERBEROS_PRINCIPAL, "HTTP/_HOST@EXAMPLE.COM"); + configuration.set(RestOptions.ADDRESS, "localhost"); + + JobManagerWebAuthenticationConfig config = + JobManagerWebAuthenticationConfig.fromConfiguration(configuration).orElseThrow(); + + assertThat(config.getKerberosPrincipal()) + .startsWith("HTTP/") + .endsWith("@EXAMPLE.COM") + .doesNotContain("_HOST"); + } + + @Test + void shouldResolveHostPlaceholderFromBindAddressWhenRestAddressIsWildcard() throws Exception { + Configuration configuration = validKerberosConfiguration(); + configuration.set(WebOptions.AUTHENTICATION_KERBEROS_PRINCIPAL, "HTTP/_HOST@EXAMPLE.COM"); + configuration.set(RestOptions.ADDRESS, "0.0.0.0"); + configuration.set(RestOptions.BIND_ADDRESS, "localhost"); + + JobManagerWebAuthenticationConfig config = + JobManagerWebAuthenticationConfig.fromConfiguration(configuration).orElseThrow(); + + assertThat(config.getKerberosPrincipal()) + .startsWith("HTTP/") + .endsWith("@EXAMPLE.COM") + .doesNotContain("_HOST") + .doesNotContain("0.0.0.0"); + } + + @Test + void shouldAllowWildcardPrincipal() throws Exception { + Configuration configuration = validKerberosConfiguration(); + configuration.set(WebOptions.AUTHENTICATION_KERBEROS_PRINCIPAL, "*"); + + JobManagerWebAuthenticationConfig config = + JobManagerWebAuthenticationConfig.fromConfiguration(configuration).orElseThrow(); + + assertThat(config.getKerberosPrincipal()).isEqualTo("*"); + } + + @Test + void shouldUseDefaultNameRules() throws Exception { + JobManagerWebAuthenticationConfig config = + JobManagerWebAuthenticationConfig.fromConfiguration(validKerberosConfiguration()) + .orElseThrow(); + + assertThat(config.getKerberosNameRules()).isEqualTo("DEFAULT"); + } + + @Test + void shouldRejectNonHttpPrincipal() throws Exception { + Configuration configuration = validKerberosConfiguration(); + configuration.set( + WebOptions.AUTHENTICATION_KERBEROS_PRINCIPAL, "flink/localhost@EXAMPLE.COM"); + + assertThatThrownBy(() -> JobManagerWebAuthenticationConfig.fromConfiguration(configuration)) + .isInstanceOf(ConfigurationException.class) + .hasMessageContaining("must start with HTTP/"); + } + + @Test + void shouldRejectInvalidCookiePath() throws Exception { + Configuration configuration = validKerberosConfiguration(); + configuration.set(WebOptions.AUTHENTICATION_COOKIE_PATH, "flink"); + + assertThatThrownBy(() -> JobManagerWebAuthenticationConfig.fromConfiguration(configuration)) + .isInstanceOf(ConfigurationException.class) + .hasMessageContaining(WebOptions.AUTHENTICATION_COOKIE_PATH.key()); + } + + @Test + void shouldRejectNonPositiveTokenValidity() throws Exception { + Configuration configuration = validKerberosConfiguration(); + configuration.set(WebOptions.AUTHENTICATION_TOKEN_VALIDITY, Duration.ZERO); + + assertThatThrownBy(() -> JobManagerWebAuthenticationConfig.fromConfiguration(configuration)) + .isInstanceOf(ConfigurationException.class) + .hasMessageContaining(WebOptions.AUTHENTICATION_TOKEN_VALIDITY.key()); + } + + @Test + void shouldRejectUnreadableKeytabPath() throws Exception { + Configuration configuration = kerberosConfiguration(); + configuration.set( + WebOptions.AUTHENTICATION_KERBEROS_PRINCIPAL, "HTTP/localhost@EXAMPLE.COM"); + configuration.set( + WebOptions.AUTHENTICATION_KERBEROS_KEYTAB, + tempDir.resolve("missing.keytab").toString()); + + assertThatThrownBy(() -> JobManagerWebAuthenticationConfig.fromConfiguration(configuration)) + .isInstanceOf(ConfigurationException.class) + .hasMessageContaining(WebOptions.AUTHENTICATION_KERBEROS_KEYTAB.key()); + } + + @Test + void shouldRejectMultipleSignatureSecretSources() throws Exception { + Configuration configuration = validKerberosConfiguration(); + configuration.set(WebOptions.AUTHENTICATION_SIGNATURE_SECRET, "secret"); + configuration.set( + WebOptions.AUTHENTICATION_SIGNATURE_SECRET_FILE, + createSecretFile("secret").toString()); + + assertThatThrownBy(() -> JobManagerWebAuthenticationConfig.fromConfiguration(configuration)) + .isInstanceOf(ConfigurationException.class) + .hasMessageContaining("Only one of"); + } + + @Test + void shouldLoadSignatureSecretFromFile() throws Exception { + Configuration configuration = validKerberosConfiguration(); + configuration.set( + WebOptions.AUTHENTICATION_SIGNATURE_SECRET_FILE, + createSecretFile("shared-secret\n").toString()); + + JobManagerWebAuthenticationConfig config = + JobManagerWebAuthenticationConfig.fromConfiguration(configuration).orElseThrow(); + + assertThat(new String(config.getSignatureSecret(), StandardCharsets.UTF_8)) + .isEqualTo("shared-secret"); + } + + private Configuration validKerberosConfiguration() throws Exception { + Configuration configuration = kerberosConfiguration(); + configuration.set( + WebOptions.AUTHENTICATION_KERBEROS_PRINCIPAL, "HTTP/localhost@EXAMPLE.COM"); + configuration.set(WebOptions.AUTHENTICATION_KERBEROS_KEYTAB, createKeytab().toString()); + return configuration; + } + + private static Configuration kerberosConfiguration() { + Configuration configuration = new Configuration(); + configuration.set(WebOptions.AUTHENTICATION_TYPE, WebAuthenticationType.KERBEROS); + return configuration; + } + + private Path createKeytab() throws Exception { + return Files.createFile(tempDir.resolve("test-" + System.nanoTime() + ".keytab")); + } + + private Path createSecretFile(String secret) throws Exception { + Path secretFile = tempDir.resolve("secret-" + System.nanoTime()); + return Files.write(secretFile, secret.getBytes(StandardCharsets.UTF_8)); + } +} diff --git a/flink-runtime/src/test/java/org/apache/flink/runtime/webmonitor/security/JobManagerWebAuthenticationHandlerTest.java b/flink-runtime/src/test/java/org/apache/flink/runtime/webmonitor/security/JobManagerWebAuthenticationHandlerTest.java new file mode 100644 index 0000000000000..aa97a91a875f6 --- /dev/null +++ b/flink-runtime/src/test/java/org/apache/flink/runtime/webmonitor/security/JobManagerWebAuthenticationHandlerTest.java @@ -0,0 +1,375 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.runtime.webmonitor.security; + +import org.apache.flink.configuration.Configuration; + +import org.apache.flink.shaded.netty4.io.netty.channel.ChannelHandler; +import org.apache.flink.shaded.netty4.io.netty.channel.ChannelHandlerContext; +import org.apache.flink.shaded.netty4.io.netty.channel.ChannelInboundHandlerAdapter; +import org.apache.flink.shaded.netty4.io.netty.channel.embedded.EmbeddedChannel; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.DefaultFullHttpRequest; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.DefaultFullHttpResponse; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.FullHttpRequest; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.FullHttpResponse; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpHeaderNames; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpMethod; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpResponseStatus; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpVersion; +import org.apache.flink.shaded.netty4.io.netty.util.ReferenceCountUtil; + +import org.apache.hadoop.security.authentication.client.AuthenticatedURL; +import org.apache.hadoop.security.authentication.client.AuthenticationException; +import org.apache.hadoop.security.authentication.server.AuthenticationToken; +import org.junit.jupiter.api.Test; + +import java.nio.charset.StandardCharsets; +import java.time.Clock; +import java.time.Duration; +import java.time.Instant; +import java.time.ZoneOffset; +import java.util.Collections; +import java.util.Map; +import java.util.Optional; +import java.util.concurrent.atomic.AtomicReference; + +import static org.assertj.core.api.Assertions.assertThat; + +/** Tests for {@link JobManagerWebAuthenticationHandler}. */ +class JobManagerWebAuthenticationHandlerTest { + + private static final byte[] SECRET = "secret".getBytes(StandardCharsets.UTF_8); + private static final Clock NOW = Clock.fixed(Instant.ofEpochMilli(10_000L), ZoneOffset.UTC); + + @Test + void shouldNotCreateHandlerWhenAuthenticationIsDisabled() throws Exception { + assertThat( + JobManagerWebAuthenticationHandler.createFactory( + new Configuration(), Collections.emptyMap())) + .isEmpty(); + } + + @Test + void shouldChallengeUnauthenticatedRequest() { + EmbeddedChannel channel = + channel( + authorization -> + JobManagerSpnegoAuthenticationResult.challenge("Negotiate")); + + channel.writeInbound(request()); + + FullHttpResponse response = channel.readOutbound(); + try { + assertThat(response.status()).isEqualTo(HttpResponseStatus.UNAUTHORIZED); + assertThat(response.headers().get(HttpHeaderNames.WWW_AUTHENTICATE)) + .isEqualTo("Negotiate"); + assertThat(response.headers().getInt(HttpHeaderNames.CONTENT_LENGTH)).isZero(); + assertThat(response.headers().getAll(HttpHeaderNames.SET_COOKIE)) + .anySatisfy( + cookie -> + assertThat(cookie) + .startsWith(AuthenticatedURL.AUTH_COOKIE + "=") + .contains("Max-Age=0") + .contains("HttpOnly")); + } finally { + response.release(); + channel.finishAndReleaseAll(); + } + } + + @Test + void shouldRejectInvalidSpnegoToken() { + EmbeddedChannel channel = + channel( + authorization -> { + throw new AuthenticationException("bad token"); + }); + + FullHttpRequest request = request(); + request.headers().set(HttpHeaderNames.AUTHORIZATION, "Negotiate invalid"); + channel.writeInbound(request); + + FullHttpResponse response = channel.readOutbound(); + try { + assertThat(response.status()).isEqualTo(HttpResponseStatus.FORBIDDEN); + assertThat(response.headers().contains(HttpHeaderNames.WWW_AUTHENTICATE)).isFalse(); + assertThat(response.headers().getAll(HttpHeaderNames.SET_COOKIE)) + .anySatisfy( + cookie -> + assertThat(cookie) + .startsWith(AuthenticatedURL.AUTH_COOKIE + "=") + .contains("Max-Age=0")); + } finally { + response.release(); + channel.finishAndReleaseAll(); + } + } + + @Test + void shouldPassThroughValidCookie() { + JobManagerAuthenticationTokenSigner signer = signer(NOW, Duration.ofHours(1)); + EmbeddedChannel channel = + channel( + authorization -> + JobManagerSpnegoAuthenticationResult.challenge("Negotiate"), + signer, + false, + Collections.emptyMap()); + + FullHttpRequest request = request(); + request.headers() + .set( + HttpHeaderNames.COOKIE, + AuthenticatedURL.AUTH_COOKIE + + "=\"" + + signer.signToken("alice", "alice@EXAMPLE.COM") + + "\""); + channel.writeInbound(request); + + FullHttpRequest forwarded = channel.readInbound(); + try { + assertThat(forwarded.uri()).isEqualTo("/jobs/overview"); + assertThat((Object) channel.readOutbound()).isNull(); + } finally { + forwarded.release(); + channel.finishAndReleaseAll(); + } + } + + @Test + void shouldExposeAuthenticatedUserWhileHandlingRequestWithValidSignedCookie() { + AtomicReference> authenticatedUser = + new AtomicReference<>(Optional.empty()); + JobManagerAuthenticationTokenSigner signer = signer(NOW, Duration.ofHours(1)); + EmbeddedChannel channel = + channel( + authorization -> + JobManagerSpnegoAuthenticationResult.challenge("Negotiate"), + signer, + false, + Collections.emptyMap(), + new ChannelInboundHandlerAdapter() { + @Override + public void channelRead(ChannelHandlerContext ctx, Object msg) { + authenticatedUser.set( + JobManagerWebAuthenticationHandler.getAuthenticatedUser( + ctx)); + ReferenceCountUtil.release(msg); + } + }); + + FullHttpRequest request = request(); + request.headers() + .set( + HttpHeaderNames.COOKIE, + AuthenticatedURL.AUTH_COOKIE + + "=\"" + + signer.signToken("alice", "alice@EXAMPLE.COM") + + "\""); + channel.writeInbound(request); + + try { + assertThat(authenticatedUser.get()) + .hasValueSatisfying( + user -> { + assertThat(user.getUserName()).isEqualTo("alice"); + assertThat(user.getPrincipal()).isEqualTo("alice@EXAMPLE.COM"); + assertThat(user.getType()) + .isEqualTo( + JobManagerAuthenticationTokenSigner + .TOKEN_TYPE_KERBEROS); + }); + assertThat( + JobManagerWebAuthenticationHandler.getAuthenticatedUser( + channel.pipeline().firstContext())) + .isEmpty(); + } finally { + channel.finishAndReleaseAll(); + } + } + + @Test + void shouldIgnoreExpiredCookieAndRestartChallenge() { + JobManagerAuthenticationTokenSigner signer = signer(NOW, Duration.ofMillis(1)); + String expiredToken = signer.signToken("alice", "alice@EXAMPLE.COM"); + EmbeddedChannel channel = + channel( + authorization -> + JobManagerSpnegoAuthenticationResult.challenge("Negotiate"), + signer(Clock.offset(NOW, Duration.ofMillis(2)), Duration.ofMillis(1)), + false, + Collections.emptyMap()); + + FullHttpRequest request = request(); + request.headers() + .set( + HttpHeaderNames.COOKIE, + AuthenticatedURL.AUTH_COOKIE + "=\"" + expiredToken + "\""); + channel.writeInbound(request); + + FullHttpResponse response = channel.readOutbound(); + try { + assertThat(response.status()).isEqualTo(HttpResponseStatus.UNAUTHORIZED); + assertThat(response.headers().get(HttpHeaderNames.WWW_AUTHENTICATE)) + .isEqualTo("Negotiate"); + } finally { + response.release(); + channel.finishAndReleaseAll(); + } + } + + @Test + void shouldIssueCookieAfterSuccessfulSpnegoAuthentication() { + AtomicReference> authenticatedUser = + new AtomicReference<>(Optional.empty()); + EmbeddedChannel channel = + channel( + authorization -> + JobManagerSpnegoAuthenticationResult.authenticated( + new AuthenticationToken( + "alice", + "alice@EXAMPLE.COM", + JobManagerAuthenticationTokenSigner + .TOKEN_TYPE_KERBEROS), + "Negotiate server-token"), + signer(NOW, Duration.ofHours(1)), + false, + Collections.emptyMap(), + new ChannelInboundHandlerAdapter() { + @Override + public void channelRead(ChannelHandlerContext ctx, Object msg) { + authenticatedUser.set( + JobManagerWebAuthenticationHandler.getAuthenticatedUser( + ctx)); + ReferenceCountUtil.release(msg); + } + }); + + FullHttpRequest request = request(); + request.headers().set(HttpHeaderNames.AUTHORIZATION, "Negotiate client-token"); + channel.writeInbound(request); + + channel.writeOutbound( + new DefaultFullHttpResponse(HttpVersion.HTTP_1_1, HttpResponseStatus.OK)); + FullHttpResponse response = channel.readOutbound(); + try { + assertThat(authenticatedUser.get()) + .hasValueSatisfying( + user -> { + assertThat(user.getUserName()).isEqualTo("alice"); + assertThat(user.getPrincipal()).isEqualTo("alice@EXAMPLE.COM"); + assertThat(user.getType()) + .isEqualTo( + JobManagerAuthenticationTokenSigner + .TOKEN_TYPE_KERBEROS); + }); + assertThat(response.headers().get(HttpHeaderNames.SET_COOKIE)) + .contains(AuthenticatedURL.AUTH_COOKIE + "=\"") + .contains("; Path=/") + .contains("; HttpOnly"); + assertThat(response.headers().get(HttpHeaderNames.WWW_AUTHENTICATE)) + .isEqualTo("Negotiate server-token"); + } finally { + response.release(); + channel.finishAndReleaseAll(); + } + } + + @Test + void shouldMarkAuthenticationCookieSecureWhenSslIsEnabled() { + EmbeddedChannel channel = + channel( + authorization -> + JobManagerSpnegoAuthenticationResult.authenticated( + new AuthenticationToken( + "alice", + "alice@EXAMPLE.COM", + JobManagerAuthenticationTokenSigner + .TOKEN_TYPE_KERBEROS), + null), + signer(NOW, Duration.ofHours(1)), + true, + Collections.emptyMap()); + + FullHttpRequest request = request(); + request.headers().set(HttpHeaderNames.AUTHORIZATION, "Negotiate client-token"); + channel.writeInbound(request); + FullHttpRequest forwarded = channel.readInbound(); + forwarded.release(); + + channel.writeOutbound( + new DefaultFullHttpResponse(HttpVersion.HTTP_1_1, HttpResponseStatus.OK)); + FullHttpResponse response = channel.readOutbound(); + try { + assertThat(response.headers().get(HttpHeaderNames.SET_COOKIE)).contains("; Secure"); + } finally { + response.release(); + channel.finishAndReleaseAll(); + } + } + + @Test + void shouldIncludeConfiguredResponseHeadersInChallenge() { + EmbeddedChannel channel = + channel( + authorization -> + JobManagerSpnegoAuthenticationResult.challenge("Negotiate"), + signer(NOW, Duration.ofHours(1)), + false, + Collections.singletonMap("Access-Control-Allow-Origin", "example.com")); + + channel.writeInbound(request()); + + FullHttpResponse response = channel.readOutbound(); + try { + assertThat(response.headers().get("Access-Control-Allow-Origin")) + .isEqualTo("example.com"); + } finally { + response.release(); + channel.finishAndReleaseAll(); + } + } + + private static EmbeddedChannel channel(SpnegoAuthenticator authenticator) { + return channel( + authenticator, signer(NOW, Duration.ofHours(1)), false, Collections.emptyMap()); + } + + private static EmbeddedChannel channel( + SpnegoAuthenticator authenticator, + JobManagerAuthenticationTokenSigner signer, + boolean secureCookie, + Map responseHeaders, + ChannelHandler... additionalHandlers) { + ChannelHandler[] handlers = new ChannelHandler[additionalHandlers.length + 1]; + handlers[0] = + new JobManagerWebAuthenticationHandler( + authenticator, signer, "/", secureCookie, responseHeaders); + System.arraycopy(additionalHandlers, 0, handlers, 1, additionalHandlers.length); + return new EmbeddedChannel(handlers); + } + + private static JobManagerAuthenticationTokenSigner signer(Clock clock, Duration tokenValidity) { + return new JobManagerAuthenticationTokenSigner(SECRET, tokenValidity, clock); + } + + private static FullHttpRequest request() { + return new DefaultFullHttpRequest(HttpVersion.HTTP_1_1, HttpMethod.GET, "/jobs/overview"); + } +} diff --git a/flink-runtime/src/test/java/org/apache/flink/runtime/webmonitor/security/JobManagerWebSpnegoITCase.java b/flink-runtime/src/test/java/org/apache/flink/runtime/webmonitor/security/JobManagerWebSpnegoITCase.java new file mode 100644 index 0000000000000..907e67094225b --- /dev/null +++ b/flink-runtime/src/test/java/org/apache/flink/runtime/webmonitor/security/JobManagerWebSpnegoITCase.java @@ -0,0 +1,290 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.runtime.webmonitor.security; + +import org.apache.flink.configuration.Configuration; +import org.apache.flink.configuration.RestOptions; +import org.apache.flink.configuration.WebOptions; +import org.apache.flink.configuration.WebOptions.WebAuthenticationType; +import org.apache.flink.runtime.rest.HttpMethodWrapper; +import org.apache.flink.runtime.rest.messages.EmptyMessageParameters; +import org.apache.flink.runtime.rest.messages.EmptyRequestBody; +import org.apache.flink.runtime.rest.messages.EmptyResponseBody; +import org.apache.flink.runtime.rest.util.TestMessageHeaders; +import org.apache.flink.runtime.rest.util.TestRestHandler; +import org.apache.flink.runtime.rest.util.TestRestServerEndpoint; +import org.apache.flink.runtime.security.KerberosUtils; +import org.apache.flink.runtime.webmonitor.RestfulGateway; +import org.apache.flink.runtime.webmonitor.TestingRestfulGateway; +import org.apache.flink.runtime.webmonitor.retriever.GatewayRetriever; + +import okhttp3.MediaType; +import okhttp3.OkHttpClient; +import okhttp3.Request; +import okhttp3.RequestBody; +import okhttp3.Response; +import org.apache.hadoop.minikdc.MiniKdc; +import org.apache.hadoop.security.authentication.client.AuthenticatedURL; +import org.apache.hadoop.security.authentication.util.KerberosUtil; +import org.ietf.jgss.GSSContext; +import org.ietf.jgss.GSSManager; +import org.ietf.jgss.GSSName; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.io.TempDir; + +import javax.security.auth.Subject; +import javax.security.auth.login.AppConfigurationEntry; +import javax.security.auth.login.LoginContext; + +import java.net.InetSocketAddress; +import java.nio.file.Files; +import java.nio.file.Path; +import java.security.PrivilegedExceptionAction; +import java.util.Base64; +import java.util.Collections; +import java.util.Properties; +import java.util.concurrent.CompletableFuture; + +import static org.assertj.core.api.Assertions.assertThat; + +/** Integration tests for JobManager web SPNEGO authentication. */ +class JobManagerWebSpnegoITCase { + + private static final MediaType JSON = MediaType.parse("application/json"); + private static final String HOST = "localhost"; + private static final String CLIENT_PRINCIPAL_NAME = "alice"; + private static final String HTTP_SERVICE_PRINCIPAL_NAME = "HTTP/" + HOST; + + @Test + void shouldRequireSpnegoAndAllowKerberosClient(@TempDir Path tempDir) throws Exception { + Properties previousProperties = rememberKerberosProperties(); + MiniKdc kdc = null; + LoginContext loginContext = null; + try { + Path kdcDir = tempDir.resolve("kdc"); + Files.createDirectories(kdcDir); + Properties kdcConf = MiniKdc.createConf(); + kdcConf.setProperty(MiniKdc.KDC_BIND_ADDRESS, HOST); + kdc = new MiniKdc(kdcConf, kdcDir.toFile()); + kdc.start(); + System.setProperty(MiniKdc.JAVA_SECURITY_KRB5_CONF, kdc.getKrb5conf().toString()); + System.setProperty("sun.security.krb5.disableReferrals", "true"); + + Path serverKeytab = tempDir.resolve("server.keytab"); + Path clientKeytab = tempDir.resolve("client.keytab"); + kdc.createPrincipal(serverKeytab.toFile(), HTTP_SERVICE_PRINCIPAL_NAME); + kdc.createPrincipal(clientKeytab.toFile(), CLIENT_PRINCIPAL_NAME); + + Configuration configuration = createKerberosConfiguration(tempDir, serverKeytab); + JobManagerWebAuthenticationHandler.Factory authenticationHandlerFactory = + JobManagerWebAuthenticationHandler.createFactory( + configuration, Collections.emptyMap()) + .orElseThrow(); + + GatewayRetriever gatewayRetriever = + () -> + CompletableFuture.completedFuture( + new TestingRestfulGateway.Builder().build()); + TestMessageHeaders + jobsOverviewHeaders = + TestMessageHeaders.emptyBuilder() + .setHttpMethod(HttpMethodWrapper.GET) + .setTargetRestEndpointURL("/jobs/overview") + .build(); + TestRestHandler< + RestfulGateway, + EmptyRequestBody, + EmptyResponseBody, + EmptyMessageParameters> + jobsOverviewHandler = + new TestRestHandler<>( + gatewayRetriever, + jobsOverviewHeaders, + CompletableFuture.completedFuture( + EmptyResponseBody.getInstance())); + + try (TestRestServerEndpoint endpoint = + TestRestServerEndpoint.builder(configuration) + .withEndpointSpecificChannelHandlerFactory( + authenticationHandlerFactory::createHandler) + .withHandler(jobsOverviewHeaders, jobsOverviewHandler) + .build()) { + endpoint.start(); + String baseUrl = getBaseUrl(endpoint); + + OkHttpClient httpClient = new OkHttpClient(); + assertSpnegoChallenge( + httpClient, new Request.Builder().url(baseUrl + "/jobs/overview")); + assertSpnegoChallenge( + httpClient, + new Request.Builder() + .url(baseUrl + "/jars/upload") + .post(RequestBody.create(JSON, "{}"))); + + loginContext = + loginFromKeytab(CLIENT_PRINCIPAL_NAME + "@" + kdc.getRealm(), clientKeytab); + try (Response jobsOverviewResponse = + spnegoRequest( + httpClient, + new Request.Builder().url(baseUrl + "/jobs/overview"), + loginContext.getSubject())) { + assertThat(jobsOverviewResponse.code()).isEqualTo(200); + assertThat(jobsOverviewResponse.header("Set-Cookie")) + .contains(AuthenticatedURL.AUTH_COOKIE + "="); + } + } + } finally { + if (loginContext != null) { + loginContext.logout(); + } + if (kdc != null) { + kdc.stop(); + } + restoreKerberosProperties(previousProperties); + } + } + + private static Configuration createKerberosConfiguration(Path tempDir, Path serverKeytab) { + Configuration configuration = new Configuration(); + configuration.set(RestOptions.BIND_PORT, "0"); + configuration.set(RestOptions.BIND_ADDRESS, HOST); + configuration.set(RestOptions.ADDRESS, HOST); + configuration.set(WebOptions.UPLOAD_DIR, tempDir.resolve("uploads").toString()); + configuration.set(WebOptions.AUTHENTICATION_TYPE, WebAuthenticationType.KERBEROS); + configuration.set(WebOptions.AUTHENTICATION_KERBEROS_PRINCIPAL, "*"); + configuration.set(WebOptions.AUTHENTICATION_KERBEROS_KEYTAB, serverKeytab.toString()); + configuration.set(WebOptions.AUTHENTICATION_SIGNATURE_SECRET, "shared-test-secret"); + return configuration; + } + + private static void assertSpnegoChallenge(OkHttpClient httpClient, Request.Builder builder) + throws Exception { + try (Response response = httpClient.newCall(builder.build()).execute()) { + assertThat(response.code()).isEqualTo(401); + assertThat(response.header("WWW-Authenticate")).isEqualTo("Negotiate"); + } + } + + private static Response spnegoRequest( + OkHttpClient httpClient, Request.Builder builder, Subject clientSubject) + throws Exception { + return Subject.doAs( + clientSubject, + (PrivilegedExceptionAction) + () -> { + GSSManager gssManager = GSSManager.getInstance(); + GSSName serverName = + gssManager.createName( + "HTTP@" + HOST, GSSName.NT_HOSTBASED_SERVICE); + GSSContext gssContext = + gssManager.createContext( + serverName, + KerberosUtil.GSS_SPNEGO_MECH_OID, + null, + GSSContext.DEFAULT_LIFETIME); + try { + gssContext.requestMutualAuth(true); + byte[] clientToken = new byte[0]; + for (int i = 0; i < 2; i++) { + byte[] outputToken = + gssContext.initSecContext( + clientToken, 0, clientToken.length); + Request request = + builder.header( + "Authorization", + "Negotiate " + + Base64.getEncoder() + .encodeToString( + outputToken)) + .build(); + Response response = httpClient.newCall(request).execute(); + if (response.code() != 401) { + return response; + } + String authenticateHeader = + response.header("WWW-Authenticate", "Negotiate"); + response.close(); + clientToken = decodeServerToken(authenticateHeader); + } + throw new IllegalStateException( + "SPNEGO authentication did not complete."); + } finally { + gssContext.dispose(); + } + }); + } + + private static byte[] decodeServerToken(String authenticateHeader) { + String prefix = "Negotiate "; + if (!authenticateHeader.startsWith(prefix)) { + return new byte[0]; + } + return Base64.getDecoder().decode(authenticateHeader.substring(prefix.length())); + } + + private static LoginContext loginFromKeytab(String principal, Path keytab) throws Exception { + LoginContext loginContext = + new LoginContext( + "client", + null, + null, + new javax.security.auth.login.Configuration() { + @Override + public AppConfigurationEntry[] getAppConfigurationEntry(String name) { + return new AppConfigurationEntry[] { + KerberosUtils.keytabEntry(keytab.toString(), principal) + }; + } + }); + loginContext.login(); + return loginContext; + } + + private static String getBaseUrl(TestRestServerEndpoint endpoint) { + InetSocketAddress serverAddress = endpoint.getServerAddress(); + return "http://" + serverAddress.getHostString() + ":" + serverAddress.getPort(); + } + + private static Properties rememberKerberosProperties() { + Properties properties = new Properties(); + remember(properties, MiniKdc.JAVA_SECURITY_KRB5_CONF); + remember(properties, "sun.security.krb5.disableReferrals"); + return properties; + } + + private static void remember(Properties properties, String key) { + String value = System.getProperty(key); + if (value != null) { + properties.setProperty(key, value); + } + } + + private static void restoreKerberosProperties(Properties properties) { + restore(properties, MiniKdc.JAVA_SECURITY_KRB5_CONF); + restore(properties, "sun.security.krb5.disableReferrals"); + } + + private static void restore(Properties properties, String key) { + if (properties.containsKey(key)) { + System.setProperty(key, properties.getProperty(key)); + } else { + System.clearProperty(key); + } + } +} diff --git a/flink-table/flink-sql-gateway/pom.xml b/flink-table/flink-sql-gateway/pom.xml index c9e035f5a9a83..7bc3098f4e8b6 100644 --- a/flink-table/flink-sql-gateway/pom.xml +++ b/flink-table/flink-sql-gateway/pom.xml @@ -60,7 +60,7 @@ org.apache.flink flink-sql-gateway-api ${project.version} - ${flink.markBundledAsOptional} + ${flink.markBundledAsOptional} org.apache.flink @@ -75,6 +75,54 @@ ${flink.markBundledAsOptional} + + org.apache.hadoop + hadoop-auth + ${flink.hadoop.version} + ${flink.markBundledAsOptional} + + + log4j + log4j + + + org.slf4j + slf4j-log4j12 + + + ch.qos.reload4j + reload4j + + + org.slf4j + slf4j-reload4j + + + io.dropwizard.metrics + metrics-core + + + org.apache.zookeeper + zookeeper + + + org.apache.curator + curator-framework + + + io.netty + netty-handler + + + + + org.apache.hadoop + hadoop-annotations + ${flink.hadoop.version} + ${flink.markBundledAsOptional} + provided + + org.apache.flink @@ -118,6 +166,26 @@ com.squareup.okhttp3 okhttp test + + + org.apache.hadoop + hadoop-minikdc + ${minikdc.version} + test + + + log4j + log4j + + + org.slf4j + slf4j-log4j12 + + + org.slf4j + slf4j-reload4j + + org.apache.flink @@ -165,6 +233,18 @@ org.apache.flink:flink-sql-gateway-api org.quartz-scheduler:quartz + org.apache.hadoop:hadoop-auth + commons-codec:commons-codec + org.apache.httpcomponents:httpclient + org.apache.httpcomponents:httpcore + org.apache.hadoop.thirdparty:hadoop-shaded-guava + org.apache.kerby:kerb-core + org.apache.kerby:kerby-pkix + org.apache.kerby:kerby-asn1 + org.apache.kerby:kerby-util + org.apache.kerby:kerb-util + org.apache.kerby:kerby-config + org.apache.kerby:kerb-crypto @@ -184,6 +264,10 @@ org.terracotta.quartz org.apache.flink.table.shaded.org.terracotta.quartz + + org.apache.hadoop + org.apache.flink.table.shaded.org.apache.hadoop + diff --git a/flink-table/flink-sql-gateway/src/main/java/org/apache/flink/table/gateway/rest/SqlGatewayRestEndpointFactory.java b/flink-table/flink-sql-gateway/src/main/java/org/apache/flink/table/gateway/rest/SqlGatewayRestEndpointFactory.java index 697b10196bafd..25d2b1038c35b 100644 --- a/flink-table/flink-sql-gateway/src/main/java/org/apache/flink/table/gateway/rest/SqlGatewayRestEndpointFactory.java +++ b/flink-table/flink-sql-gateway/src/main/java/org/apache/flink/table/gateway/rest/SqlGatewayRestEndpointFactory.java @@ -25,12 +25,21 @@ import org.apache.flink.table.gateway.api.endpoint.SqlGatewayEndpointFactory; import org.apache.flink.table.gateway.api.endpoint.SqlGatewayEndpointFactoryUtils; import org.apache.flink.table.gateway.api.utils.SqlGatewayException; +import org.apache.flink.table.gateway.rest.security.SqlGatewaySpnegoAuthenticationHandlerFactory; import java.util.HashSet; import java.util.Map; import java.util.Set; import static org.apache.flink.table.gateway.rest.util.SqlGatewayRestOptions.ADDRESS; +import static org.apache.flink.table.gateway.rest.util.SqlGatewayRestOptions.AUTHENTICATION_COOKIE_PATH; +import static org.apache.flink.table.gateway.rest.util.SqlGatewayRestOptions.AUTHENTICATION_KERBEROS_KEYTAB; +import static org.apache.flink.table.gateway.rest.util.SqlGatewayRestOptions.AUTHENTICATION_KERBEROS_NAME_RULES; +import static org.apache.flink.table.gateway.rest.util.SqlGatewayRestOptions.AUTHENTICATION_KERBEROS_PRINCIPAL; +import static org.apache.flink.table.gateway.rest.util.SqlGatewayRestOptions.AUTHENTICATION_SIGNATURE_SECRET; +import static org.apache.flink.table.gateway.rest.util.SqlGatewayRestOptions.AUTHENTICATION_SIGNATURE_SECRET_FILE; +import static org.apache.flink.table.gateway.rest.util.SqlGatewayRestOptions.AUTHENTICATION_TOKEN_VALIDITY; +import static org.apache.flink.table.gateway.rest.util.SqlGatewayRestOptions.AUTHENTICATION_TYPE; import static org.apache.flink.table.gateway.rest.util.SqlGatewayRestOptions.BIND_ADDRESS; import static org.apache.flink.table.gateway.rest.util.SqlGatewayRestOptions.BIND_PORT; import static org.apache.flink.table.gateway.rest.util.SqlGatewayRestOptions.PORT; @@ -51,6 +60,7 @@ public SqlGatewayEndpoint createSqlGatewayEndpoint(Context context) { rebuildRestEndpointOptions( context.getEndpointOptions(), context.getFlinkConfiguration().toMap()); try { + SqlGatewaySpnegoAuthenticationHandlerFactory.validateConfiguration(config); return new SqlGatewayRestEndpoint(config, context.getSqlGatewayService()); } catch (Exception e) { throw new SqlGatewayException("Cannot start the rest endpoint.", e); @@ -96,6 +106,14 @@ public Set> optionalOptions() { options.add(BIND_ADDRESS); options.add(PORT); options.add(BIND_PORT); + options.add(AUTHENTICATION_TYPE); + options.add(AUTHENTICATION_KERBEROS_PRINCIPAL); + options.add(AUTHENTICATION_KERBEROS_KEYTAB); + options.add(AUTHENTICATION_KERBEROS_NAME_RULES); + options.add(AUTHENTICATION_TOKEN_VALIDITY); + options.add(AUTHENTICATION_COOKIE_PATH); + options.add(AUTHENTICATION_SIGNATURE_SECRET); + options.add(AUTHENTICATION_SIGNATURE_SECRET_FILE); return options; } } diff --git a/flink-table/flink-sql-gateway/src/main/java/org/apache/flink/table/gateway/rest/security/SpnegoAuthenticator.java b/flink-table/flink-sql-gateway/src/main/java/org/apache/flink/table/gateway/rest/security/SpnegoAuthenticator.java new file mode 100644 index 0000000000000..ccf98c5d62483 --- /dev/null +++ b/flink-table/flink-sql-gateway/src/main/java/org/apache/flink/table/gateway/rest/security/SpnegoAuthenticator.java @@ -0,0 +1,28 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.table.gateway.rest.security; + +import org.apache.hadoop.security.authentication.client.AuthenticationException; + +/** Authentication step contract used by the Netty SPNEGO handler. */ +interface SpnegoAuthenticator { + + SqlGatewaySpnegoAuthenticationResult authenticate(String authorizationHeader) + throws AuthenticationException; +} diff --git a/flink-table/flink-sql-gateway/src/main/java/org/apache/flink/table/gateway/rest/security/SqlGatewayAuthenticationTokenSigner.java b/flink-table/flink-sql-gateway/src/main/java/org/apache/flink/table/gateway/rest/security/SqlGatewayAuthenticationTokenSigner.java new file mode 100644 index 0000000000000..8746972935706 --- /dev/null +++ b/flink-table/flink-sql-gateway/src/main/java/org/apache/flink/table/gateway/rest/security/SqlGatewayAuthenticationTokenSigner.java @@ -0,0 +1,112 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.table.gateway.rest.security; + +import org.apache.hadoop.security.authentication.client.AuthenticationException; +import org.apache.hadoop.security.authentication.server.AuthenticationToken; + +import javax.crypto.Mac; +import javax.crypto.spec.SecretKeySpec; + +import java.nio.charset.StandardCharsets; +import java.security.InvalidKeyException; +import java.security.MessageDigest; +import java.security.NoSuchAlgorithmException; +import java.time.Clock; +import java.time.Duration; +import java.util.Base64; +import java.util.Optional; + +import static org.apache.flink.util.Preconditions.checkNotNull; + +/** Signs and verifies Hadoop-compatible SQL Gateway REST authentication tokens. */ +final class SqlGatewayAuthenticationTokenSigner { + + static final String TOKEN_TYPE_KERBEROS = "kerberos"; + + private static final String SIGNATURE_SEPARATOR = "&s="; + private static final String SIGNING_ALGORITHM = "HmacSHA256"; + + private final byte[] secret; + private final Duration tokenValidity; + private final Clock clock; + + SqlGatewayAuthenticationTokenSigner(byte[] secret, Duration tokenValidity, Clock clock) { + this.secret = checkNotNull(secret).clone(); + this.tokenValidity = checkNotNull(tokenValidity); + this.clock = checkNotNull(clock); + } + + String signToken(String userName, String principal) { + AuthenticationToken token = + new AuthenticationToken(userName, principal, TOKEN_TYPE_KERBEROS); + token.setExpires(clock.millis() + tokenValidity.toMillis()); + return sign(token.toString()); + } + + Optional verifyToken(String signedToken) { + if (signedToken == null || signedToken.isEmpty()) { + return Optional.empty(); + } + try { + AuthenticationToken token = AuthenticationToken.parse(verifyAndExtract(signedToken)); + if (!TOKEN_TYPE_KERBEROS.equals(token.getType())) { + return Optional.empty(); + } + if (token.getExpires() != -1 && clock.millis() > token.getExpires()) { + return Optional.empty(); + } + return Optional.of(token); + } catch (AuthenticationException | IllegalArgumentException e) { + return Optional.empty(); + } + } + + private String sign(String rawToken) { + return rawToken + SIGNATURE_SEPARATOR + computeSignature(rawToken); + } + + private String verifyAndExtract(String signedToken) { + int index = signedToken.lastIndexOf(SIGNATURE_SEPARATOR); + if (index < 0) { + throw new IllegalArgumentException("Authentication token is not signed."); + } + + String rawToken = signedToken.substring(0, index); + String expectedSignature = computeSignature(rawToken); + String actualSignature = signedToken.substring(index + SIGNATURE_SEPARATOR.length()); + if (!MessageDigest.isEqual( + expectedSignature.getBytes(StandardCharsets.UTF_8), + actualSignature.getBytes(StandardCharsets.UTF_8))) { + throw new IllegalArgumentException("Authentication token signature is invalid."); + } + return rawToken; + } + + private String computeSignature(String value) { + try { + Mac mac = Mac.getInstance(SIGNING_ALGORITHM); + mac.init(new SecretKeySpec(secret, SIGNING_ALGORITHM)); + return Base64.getEncoder() + .encodeToString(mac.doFinal(value.getBytes(StandardCharsets.UTF_8))); + } catch (NoSuchAlgorithmException | InvalidKeyException e) { + throw new IllegalStateException("Could not sign SQL Gateway authentication token.", e); + } + } +} diff --git a/flink-table/flink-sql-gateway/src/main/java/org/apache/flink/table/gateway/rest/security/SqlGatewayRestAuthenticationConfig.java b/flink-table/flink-sql-gateway/src/main/java/org/apache/flink/table/gateway/rest/security/SqlGatewayRestAuthenticationConfig.java new file mode 100644 index 0000000000000..0e3fb4ae78c10 --- /dev/null +++ b/flink-table/flink-sql-gateway/src/main/java/org/apache/flink/table/gateway/rest/security/SqlGatewayRestAuthenticationConfig.java @@ -0,0 +1,245 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.table.gateway.rest.security; + +import org.apache.flink.configuration.Configuration; +import org.apache.flink.configuration.DelegatingConfiguration; +import org.apache.flink.configuration.SecurityOptions; +import org.apache.flink.table.gateway.api.endpoint.SqlGatewayEndpointFactoryUtils; +import org.apache.flink.table.gateway.rest.SqlGatewayRestEndpointFactory; +import org.apache.flink.table.gateway.rest.util.SqlGatewayRestOptions; +import org.apache.flink.table.gateway.rest.util.SqlGatewayRestOptions.SqlGatewayRestAuthenticationType; +import org.apache.flink.util.ConfigurationException; +import org.apache.flink.util.StringUtils; + +import java.io.IOException; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.security.SecureRandom; +import java.time.Duration; + +import static org.apache.flink.util.Preconditions.checkNotNull; + +/** Parsed SQL Gateway REST authentication configuration. */ +final class SqlGatewayRestAuthenticationConfig { + + private static final SecureRandom SECURE_RANDOM = new SecureRandom(); + + private final SqlGatewayRestAuthenticationType type; + private final String principal; + private final String keytab; + private final String nameRules; + private final Duration tokenValidity; + private final String cookiePath; + private final byte[] signatureSecret; + private final boolean secureCookie; + private final String restAddress; + + private SqlGatewayRestAuthenticationConfig( + SqlGatewayRestAuthenticationType type, + String principal, + String keytab, + String nameRules, + Duration tokenValidity, + String cookiePath, + byte[] signatureSecret, + boolean secureCookie, + String restAddress) { + this.type = checkNotNull(type); + this.principal = principal; + this.keytab = keytab; + this.nameRules = checkNotNull(nameRules); + this.tokenValidity = checkNotNull(tokenValidity); + this.cookiePath = checkNotNull(cookiePath); + this.signatureSecret = signatureSecret.clone(); + this.secureCookie = secureCookie; + this.restAddress = checkNotNull(restAddress); + } + + static SqlGatewayRestAuthenticationConfig fromConfiguration(Configuration configuration) + throws ConfigurationException { + Configuration endpointConfig = + new DelegatingConfiguration( + configuration, + SqlGatewayEndpointFactoryUtils.getSqlGatewayOptionPrefix( + SqlGatewayRestEndpointFactory.IDENTIFIER)); + + SqlGatewayRestAuthenticationType type = + endpointConfig.get(SqlGatewayRestOptions.AUTHENTICATION_TYPE); + boolean secureCookie = SecurityOptions.isRestSSLEnabled(configuration); + String restAddress = configuration.get(org.apache.flink.configuration.RestOptions.ADDRESS); + if (type == SqlGatewayRestAuthenticationType.NONE) { + return new SqlGatewayRestAuthenticationConfig( + type, + null, + null, + SqlGatewayRestOptions.AUTHENTICATION_KERBEROS_NAME_RULES.defaultValue(), + SqlGatewayRestOptions.AUTHENTICATION_TOKEN_VALIDITY.defaultValue(), + SqlGatewayRestOptions.AUTHENTICATION_COOKIE_PATH.defaultValue(), + new byte[0], + secureCookie, + restAddress); + } + + String principal = + endpointConfig.get(SqlGatewayRestOptions.AUTHENTICATION_KERBEROS_PRINCIPAL); + String keytab = endpointConfig.get(SqlGatewayRestOptions.AUTHENTICATION_KERBEROS_KEYTAB); + String nameRules = + endpointConfig.get(SqlGatewayRestOptions.AUTHENTICATION_KERBEROS_NAME_RULES); + Duration tokenValidity = + endpointConfig.get(SqlGatewayRestOptions.AUTHENTICATION_TOKEN_VALIDITY); + String cookiePath = endpointConfig.get(SqlGatewayRestOptions.AUTHENTICATION_COOKIE_PATH); + String signatureSecret = + endpointConfig.get(SqlGatewayRestOptions.AUTHENTICATION_SIGNATURE_SECRET); + String signatureSecretFile = + endpointConfig.get(SqlGatewayRestOptions.AUTHENTICATION_SIGNATURE_SECRET_FILE); + + if (!tokenValidity.isPositive()) { + throw new ConfigurationException( + String.format( + "SQL Gateway REST authentication option '%s' must be positive.", + fullKey(SqlGatewayRestOptions.AUTHENTICATION_TOKEN_VALIDITY.key()))); + } + if (StringUtils.isNullOrWhitespaceOnly(cookiePath) || !cookiePath.startsWith("/")) { + throw new ConfigurationException( + String.format( + "SQL Gateway REST authentication option '%s' must start with '/'.", + fullKey(SqlGatewayRestOptions.AUTHENTICATION_COOKIE_PATH.key()))); + } + + byte[] secret = resolveSignatureSecret(signatureSecret, signatureSecretFile); + requireNonEmpty(principal, SqlGatewayRestOptions.AUTHENTICATION_KERBEROS_PRINCIPAL.key()); + requireNonEmpty(keytab, SqlGatewayRestOptions.AUTHENTICATION_KERBEROS_KEYTAB.key()); + + return new SqlGatewayRestAuthenticationConfig( + type, + principal, + keytab, + nameRules, + tokenValidity, + cookiePath, + secret, + secureCookie, + restAddress); + } + + private static byte[] resolveSignatureSecret(String secret, String secretFile) + throws ConfigurationException { + boolean hasSecret = !StringUtils.isNullOrWhitespaceOnly(secret); + boolean hasSecretFile = !StringUtils.isNullOrWhitespaceOnly(secretFile); + if (hasSecret && hasSecretFile) { + throw new ConfigurationException( + String.format( + "Only one of SQL Gateway REST authentication options '%s' and '%s' can be configured.", + fullKey(SqlGatewayRestOptions.AUTHENTICATION_SIGNATURE_SECRET.key()), + fullKey( + SqlGatewayRestOptions.AUTHENTICATION_SIGNATURE_SECRET_FILE + .key()))); + } + if (hasSecret) { + return secret.getBytes(StandardCharsets.UTF_8); + } + if (hasSecretFile) { + try { + String fileSecret = + Files.readString(Path.of(secretFile), StandardCharsets.UTF_8).trim(); + if (fileSecret.isEmpty()) { + throw new ConfigurationException( + String.format( + "SQL Gateway REST authentication option '%s' points to an empty file.", + fullKey( + SqlGatewayRestOptions + .AUTHENTICATION_SIGNATURE_SECRET_FILE + .key()))); + } + return fileSecret.getBytes(StandardCharsets.UTF_8); + } catch (IOException e) { + throw new ConfigurationException( + String.format( + "Could not read SQL Gateway REST authentication secret file configured by '%s'.", + fullKey( + SqlGatewayRestOptions.AUTHENTICATION_SIGNATURE_SECRET_FILE + .key())), + e); + } + } + + byte[] randomSecret = new byte[32]; + SECURE_RANDOM.nextBytes(randomSecret); + return randomSecret; + } + + private static void requireNonEmpty(String value, String localKey) + throws ConfigurationException { + if (StringUtils.isNullOrWhitespaceOnly(value)) { + throw new ConfigurationException( + String.format( + "SQL Gateway REST authentication option '%s' is required when '%s' is KERBEROS.", + fullKey(localKey), + fullKey(SqlGatewayRestOptions.AUTHENTICATION_TYPE.key()))); + } + } + + private static String fullKey(String localKey) { + return SqlGatewayEndpointFactoryUtils.getSqlGatewayOptionPrefix( + SqlGatewayRestEndpointFactory.IDENTIFIER) + + localKey; + } + + SqlGatewayRestAuthenticationType type() { + return type; + } + + boolean isKerberosEnabled() { + return type == SqlGatewayRestAuthenticationType.KERBEROS; + } + + String principal() { + return principal; + } + + String keytab() { + return keytab; + } + + String nameRules() { + return nameRules; + } + + Duration tokenValidity() { + return tokenValidity; + } + + String cookiePath() { + return cookiePath; + } + + byte[] signatureSecret() { + return signatureSecret.clone(); + } + + boolean secureCookie() { + return secureCookie; + } + + String restAddress() { + return restAddress; + } +} diff --git a/flink-table/flink-sql-gateway/src/main/java/org/apache/flink/table/gateway/rest/security/SqlGatewaySpnegoAuthenticationHandler.java b/flink-table/flink-sql-gateway/src/main/java/org/apache/flink/table/gateway/rest/security/SqlGatewaySpnegoAuthenticationHandler.java new file mode 100644 index 0000000000000..cd0be41a0d1b4 --- /dev/null +++ b/flink-table/flink-sql-gateway/src/main/java/org/apache/flink/table/gateway/rest/security/SqlGatewaySpnegoAuthenticationHandler.java @@ -0,0 +1,206 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.table.gateway.rest.security; + +import org.apache.flink.shaded.netty4.io.netty.buffer.Unpooled; +import org.apache.flink.shaded.netty4.io.netty.channel.ChannelDuplexHandler; +import org.apache.flink.shaded.netty4.io.netty.channel.ChannelHandlerContext; +import org.apache.flink.shaded.netty4.io.netty.channel.ChannelPromise; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.DefaultFullHttpResponse; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.FullHttpRequest; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.FullHttpResponse; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpHeaderNames; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpHeaderValues; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpHeaders; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpResponse; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpResponseStatus; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpVersion; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.cookie.Cookie; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.cookie.ServerCookieDecoder; +import org.apache.flink.shaded.netty4.io.netty.util.AttributeKey; +import org.apache.flink.shaded.netty4.io.netty.util.ReferenceCountUtil; + +import org.apache.hadoop.security.authentication.client.AuthenticationException; +import org.apache.hadoop.security.authentication.server.AuthenticationToken; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import javax.annotation.Nullable; + +import java.util.List; +import java.util.Map; +import java.util.Optional; +import java.util.Set; + +import static org.apache.flink.util.Preconditions.checkNotNull; + +/** Netty handler that enforces SQL Gateway REST SPNEGO authentication. */ +final class SqlGatewaySpnegoAuthenticationHandler extends ChannelDuplexHandler { + + static final String HADOOP_AUTH_COOKIE = "hadoop.auth"; + + private static final Logger LOG = + LoggerFactory.getLogger(SqlGatewaySpnegoAuthenticationHandler.class); + private static final AttributeKey RESPONSE_AUTHENTICATION_ATTRIBUTE = + AttributeKey.valueOf("sql-gateway-rest-spnego-response-authentication"); + + private final SpnegoAuthenticator authenticator; + private final SqlGatewayAuthenticationTokenSigner tokenSigner; + private final String cookiePath; + private final boolean secureCookie; + private final Map responseHeaders; + + SqlGatewaySpnegoAuthenticationHandler( + SpnegoAuthenticator authenticator, + SqlGatewayAuthenticationTokenSigner tokenSigner, + String cookiePath, + boolean secureCookie, + Map responseHeaders) { + this.authenticator = checkNotNull(authenticator); + this.tokenSigner = checkNotNull(tokenSigner); + this.cookiePath = checkNotNull(cookiePath); + this.secureCookie = secureCookie; + this.responseHeaders = checkNotNull(responseHeaders); + } + + @Override + public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception { + if (!(msg instanceof FullHttpRequest)) { + ctx.fireChannelRead(msg); + return; + } + + FullHttpRequest request = (FullHttpRequest) msg; + Optional cookieToken = getAuthenticationTokenFromCookie(request); + if (cookieToken.isPresent()) { + ctx.fireChannelRead(msg); + return; + } + + try { + SqlGatewaySpnegoAuthenticationResult result = + authenticator.authenticate( + request.headers().get(HttpHeaderNames.AUTHORIZATION)); + if (!result.isAuthenticated()) { + sendResponse( + ctx, request, HttpResponseStatus.UNAUTHORIZED, result.authenticateHeader()); + return; + } + + AuthenticationToken authenticationToken = result.authenticationToken(); + String signedToken = + tokenSigner.signToken( + authenticationToken.getUserName(), authenticationToken.getName()); + ctx.channel() + .attr(RESPONSE_AUTHENTICATION_ATTRIBUTE) + .set(new ResponseAuthentication(signedToken, result.authenticateHeader())); + ctx.fireChannelRead(msg); + } catch (AuthenticationException e) { + LOG.debug("SQL Gateway REST SPNEGO authentication failed.", e); + sendResponse(ctx, request, HttpResponseStatus.FORBIDDEN, null); + } + } + + @Override + public void write(ChannelHandlerContext ctx, Object msg, ChannelPromise promise) + throws Exception { + if (msg instanceof HttpResponse) { + ResponseAuthentication responseAuthentication = + ctx.channel().attr(RESPONSE_AUTHENTICATION_ATTRIBUTE).getAndSet(null); + if (responseAuthentication != null) { + HttpHeaders headers = ((HttpResponse) msg).headers(); + headers.add(HttpHeaderNames.SET_COOKIE, createAuthCookie(responseAuthentication)); + if (responseAuthentication.authenticateHeader() != null) { + headers.set( + HttpHeaderNames.WWW_AUTHENTICATE, + responseAuthentication.authenticateHeader()); + } + } + } + ctx.write(msg, promise); + } + + private Optional getAuthenticationTokenFromCookie( + FullHttpRequest request) { + List cookieHeaders = request.headers().getAll(HttpHeaderNames.COOKIE); + for (String cookieHeader : cookieHeaders) { + Set cookies = ServerCookieDecoder.STRICT.decode(cookieHeader); + for (Cookie cookie : cookies) { + if (HADOOP_AUTH_COOKIE.equals(cookie.name())) { + return tokenSigner.verifyToken(cookie.value()); + } + } + } + return Optional.empty(); + } + + private void sendResponse( + ChannelHandlerContext ctx, + FullHttpRequest request, + HttpResponseStatus status, + @Nullable String authenticateHeader) { + FullHttpResponse response = + new DefaultFullHttpResponse(HttpVersion.HTTP_1_1, status, Unpooled.EMPTY_BUFFER); + responseHeaders.forEach((name, value) -> response.headers().set(name, value)); + response.headers().set(HttpHeaderNames.CONTENT_LENGTH, 0); + if (authenticateHeader != null) { + response.headers().set(HttpHeaderNames.WWW_AUTHENTICATE, authenticateHeader); + } + if (!HttpHeaderValues.CLOSE.contentEqualsIgnoreCase( + request.headers().get(HttpHeaderNames.CONNECTION))) { + response.headers().set(HttpHeaderNames.CONNECTION, HttpHeaderValues.KEEP_ALIVE); + } + ReferenceCountUtil.release(request); + ctx.writeAndFlush(response); + } + + private String createAuthCookie(ResponseAuthentication responseAuthentication) { + StringBuilder cookie = + new StringBuilder(HADOOP_AUTH_COOKIE) + .append("=\"") + .append(responseAuthentication.signedToken()) + .append("\"") + .append("; Path=") + .append(cookiePath); + if (secureCookie) { + cookie.append("; Secure"); + } + cookie.append("; HttpOnly"); + return cookie.toString(); + } + + private static final class ResponseAuthentication { + private final String signedToken; + @Nullable private final String authenticateHeader; + + private ResponseAuthentication(String signedToken, @Nullable String authenticateHeader) { + this.signedToken = signedToken; + this.authenticateHeader = authenticateHeader; + } + + private String signedToken() { + return signedToken; + } + + @Nullable + private String authenticateHeader() { + return authenticateHeader; + } + } +} diff --git a/flink-table/flink-sql-gateway/src/main/java/org/apache/flink/table/gateway/rest/security/SqlGatewaySpnegoAuthenticationHandlerFactory.java b/flink-table/flink-sql-gateway/src/main/java/org/apache/flink/table/gateway/rest/security/SqlGatewaySpnegoAuthenticationHandlerFactory.java new file mode 100644 index 0000000000000..00b6f49ecc1ad --- /dev/null +++ b/flink-table/flink-sql-gateway/src/main/java/org/apache/flink/table/gateway/rest/security/SqlGatewaySpnegoAuthenticationHandlerFactory.java @@ -0,0 +1,73 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.table.gateway.rest.security; + +import org.apache.flink.configuration.Configuration; +import org.apache.flink.runtime.io.network.netty.InboundChannelHandlerFactory; +import org.apache.flink.util.ConfigurationException; + +import org.apache.flink.shaded.netty4.io.netty.channel.ChannelHandler; + +import java.time.Clock; +import java.util.Map; +import java.util.Optional; + +/** Service-loaded Netty handler factory for SQL Gateway REST SPNEGO authentication. */ +public final class SqlGatewaySpnegoAuthenticationHandlerFactory + implements InboundChannelHandlerFactory { + + private static final int PRIORITY = 1000; + + @Override + public int priority() { + return PRIORITY; + } + + @Override + public Optional createHandler( + Configuration configuration, Map responseHeaders) + throws ConfigurationException { + SqlGatewayRestAuthenticationConfig config = + SqlGatewayRestAuthenticationConfig.fromConfiguration(configuration); + if (!config.isKerberosEnabled()) { + return Optional.empty(); + } + + SqlGatewaySpnegoAuthenticator authenticator = SqlGatewaySpnegoAuthenticator.create(config); + SqlGatewayAuthenticationTokenSigner signer = + new SqlGatewayAuthenticationTokenSigner( + config.signatureSecret(), config.tokenValidity(), Clock.systemUTC()); + return Optional.of( + new SqlGatewaySpnegoAuthenticationHandler( + authenticator, + signer, + config.cookiePath(), + config.secureCookie(), + responseHeaders)); + } + + public static void validateConfiguration(Configuration configuration) + throws ConfigurationException { + SqlGatewayRestAuthenticationConfig config = + SqlGatewayRestAuthenticationConfig.fromConfiguration(configuration); + if (config.isKerberosEnabled()) { + SqlGatewaySpnegoAuthenticator.create(config); + } + } +} diff --git a/flink-table/flink-sql-gateway/src/main/java/org/apache/flink/table/gateway/rest/security/SqlGatewaySpnegoAuthenticationResult.java b/flink-table/flink-sql-gateway/src/main/java/org/apache/flink/table/gateway/rest/security/SqlGatewaySpnegoAuthenticationResult.java new file mode 100644 index 0000000000000..6c61fd0b33e3b --- /dev/null +++ b/flink-table/flink-sql-gateway/src/main/java/org/apache/flink/table/gateway/rest/security/SqlGatewaySpnegoAuthenticationResult.java @@ -0,0 +1,61 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.table.gateway.rest.security; + +import org.apache.hadoop.security.authentication.server.AuthenticationToken; + +import javax.annotation.Nullable; + +import static org.apache.flink.util.Preconditions.checkNotNull; + +/** Result of one SPNEGO acceptor step. */ +final class SqlGatewaySpnegoAuthenticationResult { + + @Nullable private final AuthenticationToken authenticationToken; + @Nullable private final String authenticateHeader; + + private SqlGatewaySpnegoAuthenticationResult( + @Nullable AuthenticationToken authenticationToken, + @Nullable String authenticateHeader) { + this.authenticationToken = authenticationToken; + this.authenticateHeader = authenticateHeader; + } + + static SqlGatewaySpnegoAuthenticationResult authenticated( + AuthenticationToken authenticationToken, @Nullable String authenticateHeader) { + return new SqlGatewaySpnegoAuthenticationResult(authenticationToken, authenticateHeader); + } + + static SqlGatewaySpnegoAuthenticationResult challenge(@Nullable String authenticateHeader) { + return new SqlGatewaySpnegoAuthenticationResult(null, authenticateHeader); + } + + boolean isAuthenticated() { + return authenticationToken != null; + } + + AuthenticationToken authenticationToken() { + return checkNotNull(authenticationToken); + } + + @Nullable + String authenticateHeader() { + return authenticateHeader; + } +} diff --git a/flink-table/flink-sql-gateway/src/main/java/org/apache/flink/table/gateway/rest/security/SqlGatewaySpnegoAuthenticator.java b/flink-table/flink-sql-gateway/src/main/java/org/apache/flink/table/gateway/rest/security/SqlGatewaySpnegoAuthenticator.java new file mode 100644 index 0000000000000..049beef1d71fb --- /dev/null +++ b/flink-table/flink-sql-gateway/src/main/java/org/apache/flink/table/gateway/rest/security/SqlGatewaySpnegoAuthenticator.java @@ -0,0 +1,297 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.table.gateway.rest.security; + +import org.apache.flink.util.ConfigurationException; + +import org.apache.hadoop.security.authentication.client.AuthenticationException; +import org.apache.hadoop.security.authentication.server.AuthenticationToken; +import org.apache.hadoop.security.authentication.util.KerberosName; +import org.apache.hadoop.security.authentication.util.KerberosUtil; +import org.ietf.jgss.GSSContext; +import org.ietf.jgss.GSSCredential; +import org.ietf.jgss.GSSException; +import org.ietf.jgss.GSSManager; +import org.ietf.jgss.Oid; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import javax.annotation.Nullable; +import javax.security.auth.Subject; +import javax.security.auth.kerberos.KerberosKey; +import javax.security.auth.kerberos.KerberosPrincipal; +import javax.security.auth.kerberos.KeyTab; + +import java.io.File; +import java.io.IOException; +import java.net.InetAddress; +import java.net.UnknownHostException; +import java.security.Principal; +import java.security.PrivilegedActionException; +import java.security.PrivilegedExceptionAction; +import java.util.Base64; +import java.util.Locale; +import java.util.regex.Pattern; + +import static org.apache.flink.util.Preconditions.checkNotNull; + +/** Kerberos/SPNEGO acceptor for the SQL Gateway REST endpoint. */ +final class SqlGatewaySpnegoAuthenticator implements SpnegoAuthenticator { + + private static final Logger LOG = LoggerFactory.getLogger(SqlGatewaySpnegoAuthenticator.class); + + static final String AUTHENTICATION_SCHEME_NEGOTIATE = "Negotiate"; + static final String AUTHENTICATE_HEADER_NEGOTIATE = "Negotiate"; + + private static final Pattern HTTP_PRINCIPAL_PATTERN = Pattern.compile("HTTP/.*"); + + private final Subject serverSubject; + private final GSSManager gssManager; + + private SqlGatewaySpnegoAuthenticator(Subject serverSubject, GSSManager gssManager) { + this.serverSubject = checkNotNull(serverSubject); + this.gssManager = checkNotNull(gssManager); + } + + static SqlGatewaySpnegoAuthenticator create(SqlGatewayRestAuthenticationConfig config) + throws ConfigurationException { + String principal = replaceHostPattern(config.principal(), config.restAddress()); + File keytabFile = new File(config.keytab()); + if (!keytabFile.isFile()) { + throw new ConfigurationException( + "SQL Gateway REST authentication Kerberos keytab does not exist or is not a file: " + + config.keytab()); + } + + Subject serverSubject = new Subject(); + KeyTab keytab = KeyTab.getInstance(keytabFile); + serverSubject.getPrivateCredentials().add(keytab); + + if ("*".equals(principal)) { + addWildcardHttpPrincipals(serverSubject, config.keytab()); + } else { + addPrincipal(serverSubject, keytab, principal, config.keytab()); + } + + try { + KerberosName.setRules(config.nameRules()); + } catch (IllegalArgumentException e) { + throw new ConfigurationException( + "Invalid SQL Gateway REST authentication Kerberos name rules.", e); + } + + GSSManager gssManager; + try { + gssManager = + Subject.doAs( + serverSubject, + (PrivilegedExceptionAction) GSSManager::getInstance); + } catch (PrivilegedActionException e) { + throw new ConfigurationException( + "Could not initialize SQL Gateway REST SPNEGO acceptor.", e.getException()); + } + + return new SqlGatewaySpnegoAuthenticator(serverSubject, gssManager); + } + + @Override + public SqlGatewaySpnegoAuthenticationResult authenticate(String authorizationHeader) + throws AuthenticationException { + if (!hasNegotiateScheme(authorizationHeader)) { + return SqlGatewaySpnegoAuthenticationResult.challenge(AUTHENTICATE_HEADER_NEGOTIATE); + } + + String encodedClientToken = + authorizationHeader.substring(AUTHENTICATION_SCHEME_NEGOTIATE.length()).trim(); + if (encodedClientToken.isEmpty()) { + throw new AuthenticationException("Missing SPNEGO token."); + } + + byte[] clientToken; + try { + clientToken = Base64.getDecoder().decode(encodedClientToken); + } catch (IllegalArgumentException e) { + throw new AuthenticationException("Invalid SPNEGO token encoding.", e); + } + + try { + String serverPrincipal = KerberosUtil.getTokenServerName(clientToken); + if (!serverPrincipal.startsWith("HTTP/")) { + throw new AuthenticationException( + "Invalid SPNEGO server principal in client token."); + } + return Subject.doAs( + serverSubject, + (PrivilegedExceptionAction) + () -> runWithPrincipal(serverPrincipal, clientToken)); + } catch (PrivilegedActionException e) { + Throwable cause = e.getException(); + if (cause instanceof AuthenticationException) { + throw (AuthenticationException) cause; + } + throw new AuthenticationException(cause); + } catch (AuthenticationException e) { + throw e; + } catch (Exception e) { + throw new AuthenticationException(e); + } + } + + private SqlGatewaySpnegoAuthenticationResult runWithPrincipal( + String serverPrincipal, byte[] clientToken) throws GSSException, IOException { + GSSContext gssContext = null; + GSSCredential gssCredential = null; + try { + LOG.trace( + "Starting SQL Gateway REST SPNEGO step for server principal {}.", + serverPrincipal); + gssCredential = + gssManager.createCredential( + gssManager.createName( + serverPrincipal, KerberosUtil.NT_GSS_KRB5_PRINCIPAL_OID), + GSSCredential.INDEFINITE_LIFETIME, + new Oid[] { + KerberosUtil.GSS_SPNEGO_MECH_OID, KerberosUtil.GSS_KRB5_MECH_OID + }, + GSSCredential.ACCEPT_ONLY); + gssContext = gssManager.createContext(gssCredential); + byte[] serverToken = gssContext.acceptSecContext(clientToken, 0, clientToken.length); + String authenticateHeader = createAuthenticateHeader(serverToken); + + if (!gssContext.isEstablished()) { + LOG.trace("SQL Gateway REST SPNEGO step is not complete yet."); + return SqlGatewaySpnegoAuthenticationResult.challenge( + authenticateHeader == null + ? AUTHENTICATE_HEADER_NEGOTIATE + : authenticateHeader); + } + + String clientPrincipal = gssContext.getSrcName().toString(); + String userName = new KerberosName(clientPrincipal).getShortName(); + AuthenticationToken token = + new AuthenticationToken( + userName, + clientPrincipal, + SqlGatewayAuthenticationTokenSigner.TOKEN_TYPE_KERBEROS); + LOG.trace( + "SQL Gateway REST SPNEGO completed for client principal {}.", clientPrincipal); + return SqlGatewaySpnegoAuthenticationResult.authenticated(token, authenticateHeader); + } finally { + if (gssContext != null) { + gssContext.dispose(); + } + if (gssCredential != null) { + gssCredential.dispose(); + } + } + } + + @Nullable + private static String createAuthenticateHeader(byte[] serverToken) { + if (serverToken == null || serverToken.length == 0) { + return null; + } + return AUTHENTICATE_HEADER_NEGOTIATE + + " " + + Base64.getEncoder().encodeToString(serverToken); + } + + private static boolean hasNegotiateScheme(String authorizationHeader) { + return authorizationHeader != null + && authorizationHeader.length() >= AUTHENTICATION_SCHEME_NEGOTIATE.length() + && authorizationHeader.regionMatches( + true, + 0, + AUTHENTICATION_SCHEME_NEGOTIATE, + 0, + AUTHENTICATION_SCHEME_NEGOTIATE.length()) + && (authorizationHeader.length() == AUTHENTICATION_SCHEME_NEGOTIATE.length() + || Character.isWhitespace( + authorizationHeader.charAt( + AUTHENTICATION_SCHEME_NEGOTIATE.length()))); + } + + static String replaceHostPattern(String principal, String restAddress) + throws ConfigurationException { + if (!principal.contains("_HOST")) { + return principal; + } + try { + String hostName; + if (restAddress == null || restAddress.isEmpty() || "0.0.0.0".equals(restAddress)) { + hostName = KerberosUtil.getLocalHostName(); + } else { + hostName = InetAddress.getByName(restAddress).getCanonicalHostName(); + } + return principal.replace("_HOST", hostName.toLowerCase(Locale.US)); + } catch (UnknownHostException e) { + throw new ConfigurationException( + "Could not resolve _HOST in SQL Gateway REST Kerberos principal.", e); + } + } + + private static void addWildcardHttpPrincipals(Subject serverSubject, String keytab) + throws ConfigurationException { + String[] principals; + try { + principals = KerberosUtil.getPrincipalNames(keytab, HTTP_PRINCIPAL_PATTERN); + } catch (IOException e) { + throw new ConfigurationException( + "Could not read SQL Gateway REST authentication Kerberos keytab.", e); + } + if (principals.length == 0) { + throw new ConfigurationException( + "SQL Gateway REST authentication Kerberos keytab does not contain any HTTP principals."); + } + for (String principal : principals) { + addPrincipal(serverSubject, null, principal, keytab); + } + } + + private static void addPrincipal( + Subject serverSubject, KeyTab keytab, String principal, String keytabPath) + throws ConfigurationException { + try { + Principal kerberosPrincipal = new KerberosPrincipal(principal); + if (keytab != null) { + KerberosKey[] keys = keytab.getKeys((KerberosPrincipal) kerberosPrincipal); + if (keys.length == 0) { + throw new ConfigurationException( + String.format( + "SQL Gateway REST authentication Kerberos keytab '%s' does not contain principal '%s'.", + keytabPath, principal)); + } + for (KerberosKey key : keys) { + key.destroy(); + } + } + serverSubject.getPrincipals().add(kerberosPrincipal); + LOG.info( + "Using SQL Gateway REST SPNEGO keytab {} for principal {}.", + keytabPath, + principal); + } catch (IllegalArgumentException e) { + throw new ConfigurationException( + "Invalid SQL Gateway REST authentication Kerberos principal: " + principal, e); + } catch (javax.security.auth.DestroyFailedException e) { + throw new ConfigurationException( + "Could not validate SQL Gateway REST authentication Kerberos keytab.", e); + } + } +} diff --git a/flink-table/flink-sql-gateway/src/main/java/org/apache/flink/table/gateway/rest/util/SqlGatewayRestOptions.java b/flink-table/flink-sql-gateway/src/main/java/org/apache/flink/table/gateway/rest/util/SqlGatewayRestOptions.java index fc440b13523be..04e90bb167924 100644 --- a/flink-table/flink-sql-gateway/src/main/java/org/apache/flink/table/gateway/rest/util/SqlGatewayRestOptions.java +++ b/flink-table/flink-sql-gateway/src/main/java/org/apache/flink/table/gateway/rest/util/SqlGatewayRestOptions.java @@ -22,6 +22,8 @@ import org.apache.flink.configuration.ConfigOption; import org.apache.flink.table.gateway.rest.SqlGatewayRestEndpoint; +import java.time.Duration; + import static org.apache.flink.configuration.ConfigOptions.key; /** @@ -43,6 +45,12 @@ @PublicEvolving public class SqlGatewayRestOptions { + /** Authentication mechanism for the SQL Gateway REST endpoint. */ + public enum SqlGatewayRestAuthenticationType { + NONE, + KERBEROS + } + /** The address that should be used by clients to connect to the sql gateway server. */ public static final ConfigOption ADDRESS = key("address") @@ -77,4 +85,72 @@ public class SqlGatewayRestOptions { String.format( "The port that the client connects to. If %s has not been specified, then the sql gateway server will bind to this port.", BIND_PORT.key())); + + /** Authentication type used by the SQL Gateway REST endpoint. */ + public static final ConfigOption AUTHENTICATION_TYPE = + key("authentication.type") + .enumType(SqlGatewayRestAuthenticationType.class) + .defaultValue(SqlGatewayRestAuthenticationType.NONE) + .withDescription( + "The authentication type for the SQL Gateway REST endpoint. " + + "Supported values are NONE and KERBEROS."); + + /** Kerberos principal used by the SQL Gateway REST endpoint SPNEGO acceptor. */ + public static final ConfigOption AUTHENTICATION_KERBEROS_PRINCIPAL = + key("authentication.kerberos.principal") + .stringType() + .noDefaultValue() + .withDescription( + "Kerberos principal used by the SQL Gateway REST endpoint SPNEGO acceptor. " + + "The HTTP/_HOST@REALM pattern is supported, and '*' uses all HTTP principals from the keytab."); + + /** Kerberos keytab used by the SQL Gateway REST endpoint SPNEGO acceptor. */ + public static final ConfigOption AUTHENTICATION_KERBEROS_KEYTAB = + key("authentication.kerberos.keytab") + .stringType() + .noDefaultValue() + .withDescription( + "Kerberos keytab used by the SQL Gateway REST endpoint SPNEGO acceptor."); + + /** Kerberos auth-to-local rules used by the SQL Gateway REST endpoint. */ + public static final ConfigOption AUTHENTICATION_KERBEROS_NAME_RULES = + key("authentication.kerberos.name-rules") + .stringType() + .defaultValue("DEFAULT") + .withDescription( + "Kerberos auth-to-local rules used to derive the local user name from the authenticated Kerberos principal."); + + /** Validity period of the SQL Gateway REST authentication cookie. */ + public static final ConfigOption AUTHENTICATION_TOKEN_VALIDITY = + key("authentication.token.validity") + .durationType() + .defaultValue(Duration.ofHours(10)) + .withDescription( + "Validity period of the SQL Gateway REST authentication cookie."); + + /** Cookie path used for SQL Gateway REST authentication cookies. */ + public static final ConfigOption AUTHENTICATION_COOKIE_PATH = + key("authentication.cookie.path") + .stringType() + .defaultValue("/") + .withDescription( + "Cookie path used for SQL Gateway REST authentication cookies."); + + /** Shared secret used to sign SQL Gateway REST authentication cookies. */ + public static final ConfigOption AUTHENTICATION_SIGNATURE_SECRET = + key("authentication.signature.secret") + .stringType() + .noDefaultValue() + .withDescription( + "Shared secret used to sign SQL Gateway REST authentication cookies. " + + "If neither this option nor authentication.signature.secret-file is configured, a random per-process secret is used."); + + /** File containing the shared secret used to sign SQL Gateway REST authentication cookies. */ + public static final ConfigOption AUTHENTICATION_SIGNATURE_SECRET_FILE = + key("authentication.signature.secret-file") + .stringType() + .noDefaultValue() + .withDescription( + "File containing the shared secret used to sign SQL Gateway REST authentication cookies. " + + "If neither this option nor authentication.signature.secret is configured, a random per-process secret is used."); } diff --git a/flink-table/flink-sql-gateway/src/main/resources/META-INF/NOTICE b/flink-table/flink-sql-gateway/src/main/resources/META-INF/NOTICE index fd52ca62cf0ff..30f9ccec7b648 100644 --- a/flink-table/flink-sql-gateway/src/main/resources/META-INF/NOTICE +++ b/flink-table/flink-sql-gateway/src/main/resources/META-INF/NOTICE @@ -6,4 +6,16 @@ The Apache Software Foundation (http://www.apache.org/). This project bundles the following dependencies under the Apache Software License 2.0. (http://www.apache.org/licenses/LICENSE-2.0.txt) +- commons-codec:commons-codec:1.15 +- org.apache.hadoop:hadoop-auth:3.4.3.1-4.3.0-1 +- org.apache.hadoop.thirdparty:hadoop-shaded-guava:1.6.0.1-4.3.0-0 +- org.apache.httpcomponents:httpclient:4.5.13 +- org.apache.httpcomponents:httpcore:4.4.14 +- org.apache.kerby:kerb-core:2.0.3 +- org.apache.kerby:kerb-crypto:2.0.3 +- org.apache.kerby:kerb-util:2.0.3 +- org.apache.kerby:kerby-asn1:2.0.3 +- org.apache.kerby:kerby-config:2.0.3 +- org.apache.kerby:kerby-pkix:2.0.3 +- org.apache.kerby:kerby-util:2.0.3 - org.quartz-scheduler:quartz:2.3.2 diff --git a/flink-table/flink-sql-gateway/src/main/resources/META-INF/services/org.apache.flink.runtime.io.network.netty.InboundChannelHandlerFactory b/flink-table/flink-sql-gateway/src/main/resources/META-INF/services/org.apache.flink.runtime.io.network.netty.InboundChannelHandlerFactory new file mode 100644 index 0000000000000..fbcafd1f4891c --- /dev/null +++ b/flink-table/flink-sql-gateway/src/main/resources/META-INF/services/org.apache.flink.runtime.io.network.netty.InboundChannelHandlerFactory @@ -0,0 +1,16 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +org.apache.flink.table.gateway.rest.security.SqlGatewaySpnegoAuthenticationHandlerFactory diff --git a/flink-table/flink-sql-gateway/src/test/java/org/apache/flink/table/gateway/rest/security/SqlGatewayAuthenticationTokenSignerTest.java b/flink-table/flink-sql-gateway/src/test/java/org/apache/flink/table/gateway/rest/security/SqlGatewayAuthenticationTokenSignerTest.java new file mode 100644 index 0000000000000..74c8c7ad05547 --- /dev/null +++ b/flink-table/flink-sql-gateway/src/test/java/org/apache/flink/table/gateway/rest/security/SqlGatewayAuthenticationTokenSignerTest.java @@ -0,0 +1,79 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.table.gateway.rest.security; + +import org.apache.hadoop.security.authentication.server.AuthenticationToken; +import org.junit.jupiter.api.Test; + +import java.time.Clock; +import java.time.Duration; +import java.time.Instant; +import java.time.ZoneOffset; +import java.util.Optional; + +import static org.assertj.core.api.Assertions.assertThat; + +/** Tests for {@link SqlGatewayAuthenticationTokenSigner}. */ +class SqlGatewayAuthenticationTokenSignerTest { + + private static final Clock NOW = Clock.fixed(Instant.ofEpochMilli(10_000L), ZoneOffset.UTC); + + @Test + void shouldSignAndVerifyKerberosToken() { + SqlGatewayAuthenticationTokenSigner signer = + new SqlGatewayAuthenticationTokenSigner( + "secret".getBytes(), Duration.ofHours(10), NOW); + + String signedToken = signer.signToken("alice", "alice@EXAMPLE.COM"); + + Optional token = signer.verifyToken(signedToken); + assertThat(token).isPresent(); + assertThat(token.orElseThrow().getUserName()).isEqualTo("alice"); + assertThat(token.orElseThrow().getName()).isEqualTo("alice@EXAMPLE.COM"); + assertThat(token.orElseThrow().getType()) + .isEqualTo(SqlGatewayAuthenticationTokenSigner.TOKEN_TYPE_KERBEROS); + } + + @Test + void shouldRejectInvalidSignature() { + SqlGatewayAuthenticationTokenSigner signer = + new SqlGatewayAuthenticationTokenSigner( + "secret".getBytes(), Duration.ofHours(10), NOW); + + String signedToken = signer.signToken("alice", "alice@EXAMPLE.COM"); + + assertThat(signer.verifyToken(signedToken.replace("alice", "bob"))).isEmpty(); + } + + @Test + void shouldRejectExpiredToken() { + SqlGatewayAuthenticationTokenSigner issuingSigner = + new SqlGatewayAuthenticationTokenSigner( + "secret".getBytes(), Duration.ofMillis(1), NOW); + SqlGatewayAuthenticationTokenSigner verifyingSigner = + new SqlGatewayAuthenticationTokenSigner( + "secret".getBytes(), + Duration.ofMillis(1), + Clock.offset(NOW, Duration.ofMillis(2))); + + String signedToken = issuingSigner.signToken("alice", "alice@EXAMPLE.COM"); + + assertThat(verifyingSigner.verifyToken(signedToken)).isEmpty(); + } +} diff --git a/flink-table/flink-sql-gateway/src/test/java/org/apache/flink/table/gateway/rest/security/SqlGatewayRestAuthenticationConfigTest.java b/flink-table/flink-sql-gateway/src/test/java/org/apache/flink/table/gateway/rest/security/SqlGatewayRestAuthenticationConfigTest.java new file mode 100644 index 0000000000000..5624bcb16ee71 --- /dev/null +++ b/flink-table/flink-sql-gateway/src/test/java/org/apache/flink/table/gateway/rest/security/SqlGatewayRestAuthenticationConfigTest.java @@ -0,0 +1,176 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.table.gateway.rest.security; + +import org.apache.flink.configuration.Configuration; +import org.apache.flink.configuration.RestOptions; +import org.apache.flink.table.gateway.api.endpoint.SqlGatewayEndpointFactoryUtils; +import org.apache.flink.table.gateway.rest.SqlGatewayRestEndpointFactory; +import org.apache.flink.table.gateway.rest.util.SqlGatewayRestOptions; +import org.apache.flink.table.gateway.rest.util.SqlGatewayRestOptions.SqlGatewayRestAuthenticationType; +import org.apache.flink.util.ConfigurationException; + +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.io.TempDir; + +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.Collections; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.assertThatThrownBy; + +/** Tests for {@link SqlGatewayRestAuthenticationConfig}. */ +class SqlGatewayRestAuthenticationConfigTest { + + @Test + void shouldDefaultToNone() throws Exception { + SqlGatewayRestAuthenticationConfig config = + SqlGatewayRestAuthenticationConfig.fromConfiguration(baseConfiguration()); + + assertThat(config.type()).isEqualTo(SqlGatewayRestAuthenticationType.NONE); + assertThat(config.isKerberosEnabled()).isFalse(); + } + + @Test + void shouldNotInstallHandlerWhenAuthenticationIsDisabled() throws Exception { + assertThat( + new SqlGatewaySpnegoAuthenticationHandlerFactory() + .createHandler(baseConfiguration(), Collections.emptyMap())) + .isEmpty(); + } + + @Test + void shouldRequirePrincipalForKerberos() { + Configuration configuration = baseConfiguration(); + set(configuration, SqlGatewayRestOptions.AUTHENTICATION_TYPE.key(), "KERBEROS"); + set(configuration, SqlGatewayRestOptions.AUTHENTICATION_KERBEROS_KEYTAB.key(), "/tmp/a"); + + assertThatThrownBy( + () -> SqlGatewayRestAuthenticationConfig.fromConfiguration(configuration)) + .isInstanceOf(ConfigurationException.class) + .hasMessageContaining("authentication.kerberos.principal"); + } + + @Test + void shouldRequireKeytabForKerberos() { + Configuration configuration = baseConfiguration(); + set(configuration, SqlGatewayRestOptions.AUTHENTICATION_TYPE.key(), "KERBEROS"); + set( + configuration, + SqlGatewayRestOptions.AUTHENTICATION_KERBEROS_PRINCIPAL.key(), + "HTTP/localhost@EXAMPLE.COM"); + + assertThatThrownBy( + () -> SqlGatewayRestAuthenticationConfig.fromConfiguration(configuration)) + .isInstanceOf(ConfigurationException.class) + .hasMessageContaining("authentication.kerberos.keytab"); + } + + @Test + void shouldRejectBothInlineSecretAndSecretFile(@TempDir Path tempDir) throws Exception { + Path secretFile = tempDir.resolve("secret"); + Files.writeString(secretFile, "secret-file", StandardCharsets.UTF_8); + + Configuration configuration = baseConfiguration(); + set(configuration, SqlGatewayRestOptions.AUTHENTICATION_TYPE.key(), "KERBEROS"); + set( + configuration, + SqlGatewayRestOptions.AUTHENTICATION_KERBEROS_PRINCIPAL.key(), + "HTTP/localhost@EXAMPLE.COM"); + set(configuration, SqlGatewayRestOptions.AUTHENTICATION_KERBEROS_KEYTAB.key(), "/tmp/a"); + set(configuration, SqlGatewayRestOptions.AUTHENTICATION_SIGNATURE_SECRET.key(), "secret"); + set( + configuration, + SqlGatewayRestOptions.AUTHENTICATION_SIGNATURE_SECRET_FILE.key(), + secretFile.toString()); + + assertThatThrownBy( + () -> SqlGatewayRestAuthenticationConfig.fromConfiguration(configuration)) + .isInstanceOf(ConfigurationException.class) + .hasMessageContaining("signature.secret") + .hasMessageContaining("signature.secret-file"); + } + + @Test + void shouldReadSecretFile(@TempDir Path tempDir) throws Exception { + Path secretFile = tempDir.resolve("secret"); + Files.writeString(secretFile, "secret-file\n", StandardCharsets.UTF_8); + + Configuration configuration = kerberosConfiguration(); + set( + configuration, + SqlGatewayRestOptions.AUTHENTICATION_SIGNATURE_SECRET_FILE.key(), + secretFile.toString()); + + SqlGatewayRestAuthenticationConfig config = + SqlGatewayRestAuthenticationConfig.fromConfiguration(configuration); + + assertThat(new String(config.signatureSecret(), StandardCharsets.UTF_8)) + .isEqualTo("secret-file"); + } + + @Test + void shouldReplaceHostPattern() throws Exception { + String principal = + SqlGatewaySpnegoAuthenticator.replaceHostPattern( + "HTTP/_HOST@EXAMPLE.COM", "localhost"); + + assertThat(principal).startsWith("HTTP/"); + assertThat(principal).endsWith("@EXAMPLE.COM"); + assertThat(principal).doesNotContain("_HOST"); + } + + @Test + void shouldRejectInvalidAuthenticationType() { + Configuration configuration = baseConfiguration(); + set(configuration, SqlGatewayRestOptions.AUTHENTICATION_TYPE.key(), "BASIC"); + + assertThatThrownBy( + () -> SqlGatewayRestAuthenticationConfig.fromConfiguration(configuration)) + .isInstanceOf(IllegalArgumentException.class) + .hasMessageContaining("BASIC"); + } + + private static Configuration kerberosConfiguration() { + Configuration configuration = baseConfiguration(); + set(configuration, SqlGatewayRestOptions.AUTHENTICATION_TYPE.key(), "KERBEROS"); + set( + configuration, + SqlGatewayRestOptions.AUTHENTICATION_KERBEROS_PRINCIPAL.key(), + "HTTP/localhost@EXAMPLE.COM"); + set(configuration, SqlGatewayRestOptions.AUTHENTICATION_KERBEROS_KEYTAB.key(), "/tmp/a"); + return configuration; + } + + private static Configuration baseConfiguration() { + Configuration configuration = new Configuration(); + configuration.set(RestOptions.ADDRESS, "localhost"); + return configuration; + } + + private static void set(Configuration configuration, String localKey, String value) { + configuration.setString( + SqlGatewayEndpointFactoryUtils.getSqlGatewayOptionPrefix( + SqlGatewayRestEndpointFactory.IDENTIFIER) + + localKey, + value); + } +} diff --git a/flink-table/flink-sql-gateway/src/test/java/org/apache/flink/table/gateway/rest/security/SqlGatewaySpnegoAuthenticationHandlerTest.java b/flink-table/flink-sql-gateway/src/test/java/org/apache/flink/table/gateway/rest/security/SqlGatewaySpnegoAuthenticationHandlerTest.java new file mode 100644 index 0000000000000..fedec6d7b4a41 --- /dev/null +++ b/flink-table/flink-sql-gateway/src/test/java/org/apache/flink/table/gateway/rest/security/SqlGatewaySpnegoAuthenticationHandlerTest.java @@ -0,0 +1,206 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.table.gateway.rest.security; + +import org.apache.flink.shaded.netty4.io.netty.channel.embedded.EmbeddedChannel; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.DefaultFullHttpRequest; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.DefaultFullHttpResponse; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.FullHttpRequest; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.FullHttpResponse; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpHeaderNames; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpMethod; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpResponseStatus; +import org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpVersion; + +import org.apache.hadoop.security.authentication.client.AuthenticationException; +import org.apache.hadoop.security.authentication.server.AuthenticationToken; +import org.junit.jupiter.api.Test; + +import java.time.Clock; +import java.time.Duration; +import java.time.Instant; +import java.time.ZoneOffset; +import java.util.Collections; + +import static org.assertj.core.api.Assertions.assertThat; + +/** Tests for {@link SqlGatewaySpnegoAuthenticationHandler}. */ +class SqlGatewaySpnegoAuthenticationHandlerTest { + + private static final byte[] SECRET = "secret".getBytes(); + private static final Clock NOW = Clock.fixed(Instant.ofEpochMilli(10_000L), ZoneOffset.UTC); + + @Test + void shouldChallengeUnauthenticatedRequest() { + EmbeddedChannel channel = + channel( + authorization -> + SqlGatewaySpnegoAuthenticationResult.challenge("Negotiate")); + + channel.writeInbound(request()); + + FullHttpResponse response = channel.readOutbound(); + try { + assertThat(response.status()).isEqualTo(HttpResponseStatus.UNAUTHORIZED); + assertThat(response.headers().get(HttpHeaderNames.WWW_AUTHENTICATE)) + .isEqualTo("Negotiate"); + } finally { + response.release(); + channel.finishAndReleaseAll(); + } + } + + @Test + void shouldRejectInvalidSpnegoToken() { + EmbeddedChannel channel = + channel( + authorization -> { + throw new AuthenticationException("bad token"); + }); + + FullHttpRequest request = request(); + request.headers().set(HttpHeaderNames.AUTHORIZATION, "Negotiate invalid"); + channel.writeInbound(request); + + FullHttpResponse response = channel.readOutbound(); + try { + assertThat(response.status()).isEqualTo(HttpResponseStatus.FORBIDDEN); + assertThat(response.headers().contains(HttpHeaderNames.WWW_AUTHENTICATE)).isFalse(); + } finally { + response.release(); + channel.finishAndReleaseAll(); + } + } + + @Test + void shouldPassThroughValidCookie() { + SqlGatewayAuthenticationTokenSigner signer = signer(NOW); + String signedToken = signer.signToken("alice", "alice@EXAMPLE.COM"); + EmbeddedChannel channel = + channel( + authorization -> + SqlGatewaySpnegoAuthenticationResult.challenge("Negotiate")); + + FullHttpRequest request = request(); + request.headers() + .set( + HttpHeaderNames.COOKIE, + SqlGatewaySpnegoAuthenticationHandler.HADOOP_AUTH_COOKIE + + "=\"" + + signedToken + + "\""); + channel.writeInbound(request); + + FullHttpRequest forwarded = channel.readInbound(); + try { + assertThat(forwarded.uri()).isEqualTo("/v1/info"); + assertThat((Object) channel.readOutbound()).isNull(); + } finally { + forwarded.release(); + channel.finishAndReleaseAll(); + } + } + + @Test + void shouldIgnoreExpiredCookieAndRestartChallenge() { + String expiredToken = signer(NOW).signToken("alice", "alice@EXAMPLE.COM"); + EmbeddedChannel channel = + new EmbeddedChannel( + handler( + authorization -> + SqlGatewaySpnegoAuthenticationResult.challenge("Negotiate"), + new SqlGatewayAuthenticationTokenSigner( + SECRET, + Duration.ofMillis(1), + Clock.offset(NOW, Duration.ofMillis(2))))); + + FullHttpRequest request = request(); + request.headers() + .set( + HttpHeaderNames.COOKIE, + SqlGatewaySpnegoAuthenticationHandler.HADOOP_AUTH_COOKIE + + "=\"" + + expiredToken + + "\""); + channel.writeInbound(request); + + FullHttpResponse response = channel.readOutbound(); + try { + assertThat(response.status()).isEqualTo(HttpResponseStatus.UNAUTHORIZED); + assertThat(response.headers().get(HttpHeaderNames.WWW_AUTHENTICATE)) + .isEqualTo("Negotiate"); + } finally { + response.release(); + channel.finishAndReleaseAll(); + } + } + + @Test + void shouldIssueCookieAfterSuccessfulSpnegoAuthentication() { + EmbeddedChannel channel = + channel( + authorization -> + SqlGatewaySpnegoAuthenticationResult.authenticated( + new AuthenticationToken( + "alice", + "alice@EXAMPLE.COM", + SqlGatewayAuthenticationTokenSigner + .TOKEN_TYPE_KERBEROS), + "Negotiate server-token")); + + FullHttpRequest request = request(); + request.headers().set(HttpHeaderNames.AUTHORIZATION, "Negotiate client-token"); + channel.writeInbound(request); + FullHttpRequest forwarded = channel.readInbound(); + forwarded.release(); + + channel.writeOutbound( + new DefaultFullHttpResponse(HttpVersion.HTTP_1_1, HttpResponseStatus.OK)); + FullHttpResponse response = channel.readOutbound(); + try { + assertThat(response.headers().get(HttpHeaderNames.SET_COOKIE)) + .contains(SqlGatewaySpnegoAuthenticationHandler.HADOOP_AUTH_COOKIE + "=\"") + .contains("; Path=/") + .contains("; HttpOnly"); + assertThat(response.headers().get(HttpHeaderNames.WWW_AUTHENTICATE)) + .isEqualTo("Negotiate server-token"); + } finally { + response.release(); + channel.finishAndReleaseAll(); + } + } + + private static EmbeddedChannel channel(SpnegoAuthenticator authenticator) { + return new EmbeddedChannel(handler(authenticator, signer(NOW))); + } + + private static SqlGatewaySpnegoAuthenticationHandler handler( + SpnegoAuthenticator authenticator, SqlGatewayAuthenticationTokenSigner signer) { + return new SqlGatewaySpnegoAuthenticationHandler( + authenticator, signer, "/", false, Collections.emptyMap()); + } + + private static SqlGatewayAuthenticationTokenSigner signer(Clock clock) { + return new SqlGatewayAuthenticationTokenSigner(SECRET, Duration.ofMillis(1), clock); + } + + private static FullHttpRequest request() { + return new DefaultFullHttpRequest(HttpVersion.HTTP_1_1, HttpMethod.GET, "/v1/info"); + } +} diff --git a/flink-table/flink-sql-gateway/src/test/java/org/apache/flink/table/gateway/rest/security/SqlGatewaySpnegoITCase.java b/flink-table/flink-sql-gateway/src/test/java/org/apache/flink/table/gateway/rest/security/SqlGatewaySpnegoITCase.java new file mode 100644 index 0000000000000..43cee9b11e86d --- /dev/null +++ b/flink-table/flink-sql-gateway/src/test/java/org/apache/flink/table/gateway/rest/security/SqlGatewaySpnegoITCase.java @@ -0,0 +1,277 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.flink.table.gateway.rest.security; + +import org.apache.flink.configuration.Configuration; +import org.apache.flink.runtime.security.KerberosUtils; +import org.apache.flink.table.gateway.api.session.SessionEnvironment; +import org.apache.flink.table.gateway.api.session.SessionHandle; +import org.apache.flink.table.gateway.api.utils.MockedSqlGatewayService; +import org.apache.flink.table.gateway.rest.SqlGatewayRestEndpoint; +import org.apache.flink.table.gateway.rest.util.SqlGatewayRestOptions; + +import okhttp3.MediaType; +import okhttp3.OkHttpClient; +import okhttp3.Request; +import okhttp3.RequestBody; +import okhttp3.Response; +import org.apache.hadoop.minikdc.MiniKdc; +import org.apache.hadoop.security.authentication.util.KerberosUtil; +import org.ietf.jgss.GSSContext; +import org.ietf.jgss.GSSManager; +import org.ietf.jgss.GSSName; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.io.TempDir; + +import javax.security.auth.Subject; +import javax.security.auth.login.AppConfigurationEntry; +import javax.security.auth.login.LoginContext; + +import java.net.InetSocketAddress; +import java.nio.file.Files; +import java.nio.file.Path; +import java.security.PrivilegedExceptionAction; +import java.util.Base64; +import java.util.Properties; + +import static org.apache.flink.table.gateway.rest.util.SqlGatewayRestEndpointTestUtils.getBaseConfig; +import static org.apache.flink.table.gateway.rest.util.SqlGatewayRestEndpointTestUtils.getFlinkConfig; +import static org.apache.flink.table.gateway.rest.util.SqlGatewayRestEndpointTestUtils.getSqlGatewayRestOptionFullName; +import static org.assertj.core.api.Assertions.assertThat; + +/** Integration tests for SQL Gateway REST SPNEGO authentication. */ +class SqlGatewaySpnegoITCase { + + private static final MediaType JSON = MediaType.parse("application/json"); + private static final String HOST = "localhost"; + private static final String CLIENT_PRINCIPAL_NAME = "alice"; + private static final String HTTP_SERVICE_PRINCIPAL_NAME = "HTTP/" + HOST; + + @Test + void shouldRequireSpnegoAndAllowKerberosClient(@TempDir Path tempDir) throws Exception { + Properties previousProperties = rememberKerberosProperties(); + MiniKdc kdc = null; + SqlGatewayRestEndpoint endpoint = null; + LoginContext loginContext = null; + try { + Path kdcDir = tempDir.resolve("kdc"); + Files.createDirectories(kdcDir); + Properties kdcConf = MiniKdc.createConf(); + kdcConf.setProperty(MiniKdc.KDC_BIND_ADDRESS, HOST); + kdc = new MiniKdc(kdcConf, kdcDir.toFile()); + kdc.start(); + System.setProperty(MiniKdc.JAVA_SECURITY_KRB5_CONF, kdc.getKrb5conf().toString()); + System.setProperty("sun.security.krb5.disableReferrals", "true"); + + Path serverKeytab = tempDir.resolve("server.keytab"); + Path clientKeytab = tempDir.resolve("client.keytab"); + kdc.createPrincipal(serverKeytab.toFile(), HTTP_SERVICE_PRINCIPAL_NAME); + kdc.createPrincipal(clientKeytab.toFile(), CLIENT_PRINCIPAL_NAME); + + Configuration flinkConfig = getFlinkConfig(HOST, HOST, "0"); + setAuthOption(flinkConfig, SqlGatewayRestOptions.AUTHENTICATION_TYPE.key(), "KERBEROS"); + setAuthOption( + flinkConfig, + SqlGatewayRestOptions.AUTHENTICATION_KERBEROS_PRINCIPAL.key(), + "*"); + setAuthOption( + flinkConfig, + SqlGatewayRestOptions.AUTHENTICATION_KERBEROS_KEYTAB.key(), + serverKeytab.toString()); + setAuthOption( + flinkConfig, + SqlGatewayRestOptions.AUTHENTICATION_SIGNATURE_SECRET.key(), + "shared-test-secret"); + + Configuration endpointConfig = getBaseConfig(flinkConfig); + SqlGatewaySpnegoAuthenticationHandlerFactory.validateConfiguration(endpointConfig); + endpoint = new SqlGatewayRestEndpoint(endpointConfig, new TestingSqlGatewayService()); + endpoint.start(); + String baseUrl = getBaseUrl(endpoint); + + OkHttpClient httpClient = new OkHttpClient(); + assertSpnegoChallenge(httpClient, new Request.Builder().url(baseUrl + "/v1/info")); + assertSpnegoChallenge( + httpClient, + new Request.Builder() + .url(baseUrl + "/v1/sessions") + .post(RequestBody.create(JSON, "{}"))); + + loginContext = + loginFromKeytab(CLIENT_PRINCIPAL_NAME + "@" + kdc.getRealm(), clientKeytab); + try (Response infoResponse = + spnegoRequest( + httpClient, + new Request.Builder().url(baseUrl + "/v1/info"), + loginContext.getSubject())) { + assertThat(infoResponse.code()).isEqualTo(200); + assertThat(infoResponse.header("Set-Cookie")).contains("hadoop.auth="); + } + + try (Response openSessionResponse = + spnegoRequest( + httpClient, + new Request.Builder() + .url(baseUrl + "/v1/sessions") + .post(RequestBody.create(JSON, "{}")), + loginContext.getSubject())) { + assertThat(openSessionResponse.code()).isEqualTo(200); + assertThat(openSessionResponse.body().string()).contains("sessionHandle"); + assertThat(openSessionResponse.header("Set-Cookie")).contains("hadoop.auth="); + } + } finally { + if (loginContext != null) { + loginContext.logout(); + } + if (endpoint != null) { + endpoint.stop(); + } + if (kdc != null) { + kdc.stop(); + } + restoreKerberosProperties(previousProperties); + } + } + + private static void assertSpnegoChallenge(OkHttpClient httpClient, Request.Builder builder) + throws Exception { + try (Response response = httpClient.newCall(builder.build()).execute()) { + assertThat(response.code()).isEqualTo(401); + assertThat(response.header("WWW-Authenticate")).isEqualTo("Negotiate"); + } + } + + private static Response spnegoRequest( + OkHttpClient httpClient, Request.Builder builder, Subject clientSubject) + throws Exception { + return Subject.doAs( + clientSubject, + (PrivilegedExceptionAction) + () -> { + GSSManager gssManager = GSSManager.getInstance(); + GSSName serverName = + gssManager.createName( + "HTTP@" + HOST, GSSName.NT_HOSTBASED_SERVICE); + GSSContext gssContext = + gssManager.createContext( + serverName, + KerberosUtil.GSS_SPNEGO_MECH_OID, + null, + GSSContext.DEFAULT_LIFETIME); + try { + gssContext.requestMutualAuth(true); + byte[] clientToken = new byte[0]; + for (int i = 0; i < 2; i++) { + byte[] outputToken = + gssContext.initSecContext( + clientToken, 0, clientToken.length); + Request request = + builder.header( + "Authorization", + "Negotiate " + + Base64.getEncoder() + .encodeToString( + outputToken)) + .build(); + Response response = httpClient.newCall(request).execute(); + if (response.code() != 401) { + return response; + } + String authenticateHeader = + response.header("WWW-Authenticate", "Negotiate"); + response.close(); + clientToken = decodeServerToken(authenticateHeader); + } + throw new IllegalStateException( + "SPNEGO authentication did not complete."); + } finally { + gssContext.dispose(); + } + }); + } + + private static byte[] decodeServerToken(String authenticateHeader) { + String prefix = "Negotiate "; + if (!authenticateHeader.startsWith(prefix)) { + return new byte[0]; + } + return Base64.getDecoder().decode(authenticateHeader.substring(prefix.length())); + } + + private static LoginContext loginFromKeytab(String principal, Path keytab) throws Exception { + LoginContext loginContext = + new LoginContext( + "client", + null, + null, + new javax.security.auth.login.Configuration() { + @Override + public AppConfigurationEntry[] getAppConfigurationEntry(String name) { + return new AppConfigurationEntry[] { + KerberosUtils.keytabEntry(keytab.toString(), principal) + }; + } + }); + loginContext.login(); + return loginContext; + } + + private static String getBaseUrl(SqlGatewayRestEndpoint endpoint) { + InetSocketAddress serverAddress = endpoint.getServerAddress(); + return "http://" + serverAddress.getHostString() + ":" + serverAddress.getPort(); + } + + private static void setAuthOption(Configuration configuration, String localKey, String value) { + configuration.setString(getSqlGatewayRestOptionFullName(localKey), value); + } + + private static Properties rememberKerberosProperties() { + Properties properties = new Properties(); + remember(properties, MiniKdc.JAVA_SECURITY_KRB5_CONF); + remember(properties, "sun.security.krb5.disableReferrals"); + return properties; + } + + private static void remember(Properties properties, String key) { + String value = System.getProperty(key); + if (value != null) { + properties.setProperty(key, value); + } + } + + private static void restoreKerberosProperties(Properties properties) { + restore(properties, MiniKdc.JAVA_SECURITY_KRB5_CONF); + restore(properties, "sun.security.krb5.disableReferrals"); + } + + private static void restore(Properties properties, String key) { + if (properties.containsKey(key)) { + System.setProperty(key, properties.getProperty(key)); + } else { + System.clearProperty(key); + } + } + + private static final class TestingSqlGatewayService extends MockedSqlGatewayService { + @Override + public SessionHandle openSession(SessionEnvironment environment) { + return SessionHandle.create(); + } + } +} diff --git a/pom.xml b/pom.xml index 5428b2d1a886a..831910b373344 100644 --- a/pom.xml +++ b/pom.xml @@ -417,6 +417,18 @@ under the License. org.slf4j slf4j-log4j12 + + ch.qos.reload4j + reload4j + + + org.slf4j + slf4j-reload4j + + + io.dropwizard.metrics + metrics-core +