1. Privilege
Index
Authorization modes
This is the design document for the original Hive authorization mode . See Authorization for an overview of authorization modes, which include storage based authorization and SQL standards based authorization.
1.1 Access Privilege
Admin privilege, DB privilege, Table level privilege, column level privilege
1.1.1 Admin privileges are global privileges, and are used to perform administration.
1.1.2 DB privileges are database specific, and apply to all objects inside that database.
1.1.3 Table privileges apply to table/view/index in a given database
1.1.4 Column privileges apply to column level.
All DB/Table/Column privilege differentiate read and write privileges even though now hive does not support column level overwrite. And there is no partition level privilege.
2. Hive Operations
create index/drop index
create database/drop database
create table/drop table
create view/drop view
alter table
show databases
lock table/unlock table/show lock
add partition
archive
Select
insert overwrite directory
insert overwrite table
others include "create table as ", "create table like" etc
3. Metadata
Store the privilege information in the new metastore tables 'user', 'db', 'tables_priv', 'columns_priv'.
The user table indicates user's global privileges, which apply to all databases.
The db table determine database level access privileges, which apply to all objects inside that database.
3.1 user, group, and roles
User can belong to some groups. The group information is provided by authenticator.
And each user or group can have some privileges and roles. A role can be a member of another role, but not in a circular manner.
So hive metadata needs to store:
1) roles -> privileges, roles mapping
2) Hive user/group -> privileges, role mapping
3.1.1 Role management
create role
drop role
grant a role to a user
revoke a role from a user
3.1.2 role metadata
role_name - string
create_time - int
3.1.3 hive role user membership table
role_name - string
user_name - string
is_group – is the user name a group name
is_role – is the user name a role name
3.2 Privileges to be supported by Hive
3.2.1 metadata
The below shows how we store the grant information in metastore. The deny information is stored in a same matter (just in different tables).
So for each grant table, there will also be a deny table. The metastore tables are
user, deny_user, db, deny_db, tables_priv, deny_tables_priv, columns_priv, deny_columns_priv
Another way to do it is to add a column in the grant table to record this row is grant or deny.
We store privileges in one column, and use comma to separate different privileges.
hive> desc user;
Field
- - - -
User
isRole
isGroup
isSuper
db_priv – set (Select_priv, Insert_priv, Create_priv, Drop_priv, Reload_priv,
Grant_priv, Index_priv, Alter_priv, Show_db_priv,
Lock_tables_priv, Create_view_priv, Show_view_priv)
hive> desc db;
Field
- - - -
Db
User
isRole
isGroup
Table_priv – set (Select_priv, Insert_priv, Create_priv, Drop_priv, Grant_priv,
Index_priv, Reload_priv, Alter_priv, Create_tmp_table_priv,
Lock_tables_priv, Create_view_priv, Show_view_priv)
hive> desc tables_priv;
Field
- - - -
Db
User
isRole
isGroup
Table_name
Grantor
Timestamp
Table_priv – set('Select','Insert',,'Create','Drop','Grant','Index','Alter','Create View','Show view')
Column_priv – set('Select','Insert',)
mysql> desc columns_priv;
Field
- - - -
Db
User
isRole
isGroup
Table_name
Column_name
Timestamp
Column_priv – set('Select','Insert','Update')
4. grant/revoke access privilege
4.1 Privilege names/types:
ALL Privileges
ALTER
Create
Create view
Delete
Drop
Index
Insert
Lock Tables
Select
Show databases
Super
4.2 show grant
4.3 grant/revoke statement
GRANT
priv_type [(column_list)]
[, priv_type [(column_list)]] ...
ON [object_type] priv_level
TO user [, user] ...
WITH ADMIN OPTION
object_type:
TABLE
priv_level:
*
| *.*
| db_name.*
| db_name.tbl_name
| tbl_name
REVOKE
priv_type [(column_list)]
[, priv_type [(column_list)]] ...
ON [object_type] priv_level
FROM user [, user] ...
REVOKE ALL PRIVILEGES, GRANT OPTION
FROM user [, user] ...
DENY
priv_type [(column_list)]
[, priv_type [(column_list)]] ...
ON [object_type] priv_level
FROM user [, user] ...
5. Authorization verification
5.1 USER/GROUP/ROLE
USER
GROUP
ROLE
GROUP is very similar to a role. And we support Group is because we may need to pass the group information to HDFS/Map-reduce.
A role can also contain other roles and privileges - and they can be granted to users and groups.
Role can be nested but not circular.
5.2 The verification steps
When a user logins to the system, he has a user name, one or few groups that he belongs to.
So it is
[
username, list of group names, list of privileges and roles that has been directly granted, list of privileges and roles that been directly granted to groups that users belongs to
].
- Steps to authorize one access: *
First try user name: # If there is an entry in 'user' that accept this access, return ACCEPT 2. If there is an entry in 'db' that accept this access, return ACCEPT 3. If there is an entry in 'table' that accept this access, return ACCEPT 4. If there is an entry in 'column' that accept this access, return ACCEPT Second try the user's group/role names one by one until we get an ACCEPT. For each role/group, we do the same routine as we did for user name.
5.3 Examples
5.3.1 I want to grant everyone (new people may join at anytime) to
db_name.*, and then later i want to protect one table db_name.T from ALL
users but a few
1) Add all users to a group 'users'. (assumption: new users will
automatically join this group). And grant 'users' ALL privileges to db_name.*
2) Add those few users to a new group 'users2'. AND REMOVE them from 'users'
3) DENY 'users' to db_name.T
4) Grant ALL on db_name.T to users2
5.3.2 I want to protect one table db_name.T from one/few users, but all
other people can access it
1) Add all users to a group 'users'. (assumption: new users will automatically
join this group). And grant 'users' ALL privileges to db_name.*.
2) Add those few users to a new group 'users2'. (Note: those few users will now
belong to 2 groups: users and user2)
3) DENY 'users2' to db_name.T
6. Where to add authorization in Hive
CliDriver and HiveServer. Basically they share the same code. If HiveServer invokes CliDriver, we can just add it into CliDriver. And we also need to make HiveServer be able to support multiple user/connections.
This still does not solve the problem if someone accesses the metastore directly (without going through Hive).
7. Implementation
7.1 Authenticator interface
We only get the user's user name, group names from the authenticator. The authenticator implementations need to provide these information. This is the only interface between authenticator and authorization.
7.2 Authorization
Authorization decision manager manages a set of authorization provider, and each provider can decide to accept or deny. And it is the decision manager to do the final decision. Can be vote based, or one -1 then deny, or one +1 then accept. Authorization provider decides whether to accept or deny an access based on his own information.
8. Metastore upgrade script for mysql
--
-- Table structure for table {{ROLES}}
--
DROP TABLE IF EXISTS {{ROLES}};
CREATE TABLE {{ROLES}} (
{{ROLE_ID}} bigint(20) NOT NULL,
{{CREATE_TIME}} int(11) NOT NULL,
{{OWNER_NAME}} varchar(128) character set latin1 collate latin1_bin default NULL,
{{ROLE_NAME}} varchar(128) character set latin1 collate latin1_bin default NULL,
PRIMARY KEY ({{ROLE_ID}}),
UNIQUE KEY {{ROLEENTITYINDEX}} ({{ROLE_NAME}})
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
--
-- Table structure for table {{ROLE_MAP}}
--
DROP TABLE IF EXISTS {{ROLE_MAP}};
CREATE TABLE {{ROLE_MAP}} (
{{ROLE_GRANT_ID}} bigint(20) NOT NULL,
{{ADD_TIME}} int(11) NOT NULL,
{{GRANT_OPTION}} smallint(6) NOT NULL,
{{GRANTOR}} varchar(128) character set latin1 collate latin1_bin default NULL,
{{GRANTOR_TYPE}} varchar(128) character set latin1 collate latin1_bin default NULL,
{{PRINCIPAL_NAME}} varchar(128) character set latin1 collate latin1_bin default NULL,
{{PRINCIPAL_TYPE}} varchar(128) character set latin1 collate latin1_bin default NULL,
{{ROLE_ID}} bigint(20) default NULL,
PRIMARY KEY ({{ROLE_GRANT_ID}}),
UNIQUE KEY {{USERROLEMAPINDEX}} ({{PRINCIPAL_NAME}},{{ROLE_ID}},{{GRANTOR}},{{GRANTOR_TYPE}}),
KEY {{ROLE_MAP_N49}} ({{ROLE_ID}}),
CONSTRAINT {{ROLE_MAP_FK1}} FOREIGN KEY ({{ROLE_ID}}) REFERENCES {{ROLES}} ({{ROLE_ID}})
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
--
-- Table structure for table {{GLOBAL_PRIVS}}
--
DROP TABLE IF EXISTS {{GLOBAL_PRIVS}};
CREATE TABLE {{GLOBAL_PRIVS}} (
{{USER_GRANT_ID}} bigint(20) NOT NULL,
{{CREATE_TIME}} int(11) NOT NULL,
{{GRANT_OPTION}} smallint(6) NOT NULL,
{{GRANTOR}} varchar(128) character set latin1 collate latin1_bin default NULL,
{{GRANTOR_TYPE}} varchar(128) character set latin1 collate latin1_bin default NULL,
{{PRINCIPAL_NAME}} varchar(128) character set latin1 collate latin1_bin default NULL,
{{PRINCIPAL_TYPE}} varchar(128) character set latin1 collate latin1_bin default NULL,
{{USER_PRIV}} varchar(128) character set latin1 collate latin1_bin default NULL,
PRIMARY KEY ({{USER_GRANT_ID}}),
UNIQUE KEY {{GLOBALPRIVILEGEINDEX}} ({{PRINCIPAL_NAME}},{{PRINCIPAL_TYPE}},{{USER_PRIV}},{{GRANTOR}},{{GRANTOR_TYPE}})
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
--
-- Table structure for table {{DB_PRIVS}}
--
DROP TABLE IF EXISTS {{DB_PRIVS}};
CREATE TABLE {{DB_PRIVS}} (
{{DB_GRANT_ID}} bigint(20) NOT NULL,
{{CREATE_TIME}} int(11) NOT NULL,
{{DB_ID}} bigint(20) default NULL,
{{GRANT_OPTION}} smallint(6) NOT NULL,
{{GRANTOR}} varchar(128) character set latin1 collate latin1_bin default NULL,
{{GRANTOR_TYPE}} varchar(128) character set latin1 collate latin1_bin default NULL,
{{PRINCIPAL_NAME}} varchar(128) character set latin1 collate latin1_bin default NULL,
{{PRINCIPAL_TYPE}} varchar(128) character set latin1 collate latin1_bin default NULL,
{{DB_PRIV}} varchar(128) character set latin1 collate latin1_bin default NULL,
PRIMARY KEY ({{DB_GRANT_ID}}),
UNIQUE KEY {{DBPRIVILEGEINDEX}} ({{DB_ID}},{{PRINCIPAL_NAME}},{{PRINCIPAL_TYPE}},{{DB_PRIV}},{{GRANTOR}},{{GRANTOR_TYPE}}),
KEY {{DB_PRIVS_N49}} ({{DB_ID}}),
CONSTRAINT {{DB_PRIVS_FK1}} FOREIGN KEY ({{DB_ID}}) REFERENCES {{DBS}} ({{DB_ID}})
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
--
-- Table structure for table {{TBL_PRIVS}}
--
DROP TABLE IF EXISTS {{TBL_PRIVS}};
CREATE TABLE {{TBL_PRIVS}} (
{{TBL_GRANT_ID}} bigint(20) NOT NULL,
{{CREATE_TIME}} int(11) NOT NULL,
{{GRANT_OPTION}} smallint(6) NOT NULL,
{{GRANTOR}} varchar(128) character set latin1 collate latin1_bin default NULL,
{{GRANTOR_TYPE}} varchar(128) character set latin1 collate latin1_bin default NULL,
{{PRINCIPAL_NAME}} varchar(128) character set latin1 collate latin1_bin default NULL,
{{PRINCIPAL_TYPE}} varchar(128) character set latin1 collate latin1_bin default NULL,
{{TBL_PRIV}} varchar(128) character set latin1 collate latin1_bin default NULL,
{{TBL_ID}} bigint(20) default NULL,
PRIMARY KEY ({{TBL_GRANT_ID}}),
KEY {{TBL_PRIVS_N49}} ({{TBL_ID}}),
KEY {{TABLEPRIVILEGEINDEX}} ({{TBL_ID}},{{PRINCIPAL_NAME}},{{PRINCIPAL_TYPE}},{{TBL_PRIV}},{{GRANTOR}},{{GRANTOR_TYPE}}),
CONSTRAINT {{TBL_PRIVS_FK1}} FOREIGN KEY ({{TBL_ID}}) REFERENCES {{TBLS}} ({{TBL_ID}})
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
--
-- Table structure for table {{PART_PRIVS}}
--
DROP TABLE IF EXISTS {{PART_PRIVS}};
CREATE TABLE {{PART_PRIVS}} (
{{PART_GRANT_ID}} bigint(20) NOT NULL,
{{CREATE_TIME}} int(11) NOT NULL,
{{GRANT_OPTION}} smallint(6) NOT NULL,
{{GRANTOR}} varchar(128) character set latin1 collate latin1_bin default NULL,
{{GRANTOR_TYPE}} varchar(128) character set latin1 collate latin1_bin default NULL,
{{PART_ID}} bigint(20) default NULL,
{{PRINCIPAL_NAME}} varchar(128) character set latin1 collate latin1_bin default NULL,
{{PRINCIPAL_TYPE}} varchar(128) character set latin1 collate latin1_bin default NULL,
{{PART_PRIV}} varchar(128) character set latin1 collate latin1_bin default NULL,
PRIMARY KEY ({{PART_GRANT_ID}}),
KEY {{PARTPRIVILEGEINDEX}} ({{PART_ID}},{{PRINCIPAL_NAME}},{{PRINCIPAL_TYPE}},{{PART_PRIV}},{{GRANTOR}},{{GRANTOR_TYPE}}),
KEY {{PART_PRIVS_N49}} ({{PART_ID}}),
CONSTRAINT {{PART_PRIVS_FK1}} FOREIGN KEY ({{PART_ID}}) REFERENCES {{PARTITIONS}} ({{PART_ID}})
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
--
-- Table structure for table {{TBL_COL_PRIVS}}
--
DROP TABLE IF EXISTS {{TBL_COL_PRIVS}};
CREATE TABLE {{TBL_COL_PRIVS}} (
{{TBL_COLUMN_GRANT_ID}} bigint(20) NOT NULL,
{{COLUMN_NAME}} varchar(128) character set latin1 collate latin1_bin default NULL,
{{CREATE_TIME}} int(11) NOT NULL,
{{GRANT_OPTION}} smallint(6) NOT NULL,
{{GRANTOR}} varchar(128) character set latin1 collate latin1_bin default NULL,
{{GRANTOR_TYPE}} varchar(128) character set latin1 collate latin1_bin default NULL,
{{PRINCIPAL_NAME}} varchar(128) character set latin1 collate latin1_bin default NULL,
{{PRINCIPAL_TYPE}} varchar(128) character set latin1 collate latin1_bin default NULL,
{{TBL_COL_PRIV}} varchar(128) character set latin1 collate latin1_bin default NULL,
{{TBL_ID}} bigint(20) default NULL,
PRIMARY KEY ({{TBL_COLUMN_GRANT_ID}}),
KEY {{TABLECOLUMNPRIVILEGEINDEX}} ({{TBL_ID}},{{COLUMN_NAME}},{{PRINCIPAL_NAME}},{{PRINCIPAL_TYPE}},{{TBL_COL_PRIV}},{{GRANTOR}},{{GRANTOR_TYPE}}),
KEY {{TBL_COL_PRIVS_N49}} ({{TBL_ID}}),
CONSTRAINT {{TBL_COL_PRIVS_FK1}} FOREIGN KEY ({{TBL_ID}}) REFERENCES {{TBLS}} ({{TBL_ID}})
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
DROP TABLE IF EXISTS {{PART_COL_PRIVS}};
CREATE TABLE {{PART_COL_PRIVS}} (
{{PART_COLUMN_GRANT_ID}} bigint(20) NOT NULL,
{{COLUMN_NAME}} varchar(128) character set latin1 collate latin1_bin default NULL,
{{CREATE_TIME}} int(11) NOT NULL,
{{GRANT_OPTION}} smallint(6) NOT NULL,
{{GRANTOR}} varchar(128) character set latin1 collate latin1_bin default NULL,
{{GRANTOR_TYPE}} varchar(128) character set latin1 collate latin1_bin default NULL,
{{PART_ID}} bigint(20) default NULL,
{{PRINCIPAL_NAME}} varchar(128) character set latin1 collate latin1_bin default NULL,
{{PRINCIPAL_TYPE}} varchar(128) character set latin1 collate latin1_bin default NULL,
{{PART_COL_PRIV}} varchar(128) character set latin1 collate latin1_bin default NULL,
PRIMARY KEY ({{PART_COLUMN_GRANT_ID}}),
KEY {{PART_COL_PRIVS_N49}} ({{PART_ID}}),
KEY {{PARTITIONCOLUMNPRIVILEGEINDEX}} ({{PART_ID}},{{COLUMN_NAME}},{{PRINCIPAL_NAME}},{{PRINCIPAL_TYPE}},{{PART_COL_PRIV}},{{GRANTOR}},{{GRANTOR_TYPE}}),
CONSTRAINT {{PART_COL_PRIVS_FK1}} FOREIGN KEY ({{PART_ID}}) REFERENCES {{PARTITIONS}} ({{PART_ID}})
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
HDFS Permission
The above has a STRONG assumption on the file layer security. Users can easily by-pass the security if the hdfs file permission is open to him. We hope we can easily plug in external authorizations (like HDFS permission/Howl permission) to alter the authorization result or even the rule.
