...
Return Type | Name(Signature) | Description | ||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
int | ascii(string str) | Returns the numeric value of the first character of str. | ||||||||||||||
string | base64(binary bin) | Converts the argument from binary to a base64 string. | ||||||||||||||
int | character_length(string str) | Returns the number of UTF-8 characters contained in str. The function char_length is shorthand for this function. | ||||||||||||||
string | chr(bigint|double A) | Returns the ASCII character having the binary equivalent to A. If A is larger than 256 the result is equivalent to chr(A % 256). Example: select chr(88); returns "X". | ||||||||||||||
string | concat(string|binary A, string|binary B...) | Returns the string or bytes resulting from concatenating the strings or bytes passed in as parameters in order. For example, concat('foo', 'bar') results in 'foobar'. Note that this function can take any number of input strings. | ||||||||||||||
array<struct<string,double>> | context_ngrams(array<array<string>>, array<string>, int K, int pf) | Returns the top-k contextual N-grams from a set of tokenized sentences, given a string of "context". See StatisticsAndDataMining for more information. | ||||||||||||||
string | concat_ws(string SEP, string A, string B...) | Like concat() above, but with custom separator SEP. | ||||||||||||||
string | concat_ws(string SEP, array<string>) | Like concat_ws() above, but taking an array of strings. | ||||||||||||||
string | decode(binary bin, string charset) | Decodes the first argument into a String using the provided character set (one of 'US-ASCII', 'ISO-8859-1', 'UTF-8', 'UTF-16BE', 'UTF-16LE', 'UTF-16'). If either argument is null, the result will also be null. | ||||||||||||||
string | elt(N int,str1 string,str2 string,str3 string,...) | Return string at index number. For example elt(2,'hello','world') returns 'world'. Returns NULL if N is less than 1 or greater than the number of arguments. | ||||||||||||||
binary | encode(string src, string charset) | Encodes the first argument into a BINARY using the provided character set (one of 'US-ASCII', 'ISO-8859-1', 'UTF-8', 'UTF-16BE', 'UTF-16LE', 'UTF-16'). If either argument is null, the result will also be null. | ||||||||||||||
int | field(val T,val1 T,val2 T,val3 T,...) | Returns the index of val in the val1,val2,val3,... list or 0 if not found. For example field('world','say','hello','world') returns 3. | ||||||||||||||
int | find_in_set(string str, string strList) | Returns the first occurance of str in strList where strList is a comma-delimited string. Returns null if either argument is null. Returns 0 if the first argument contains any commas. For example, find_in_set('ab', 'abc,b,ab,c,def') returns 3. | ||||||||||||||
string | format_number(number x, int d) | Formats the number X to a format like '#,###,###.##', rounded to D decimal places, and returns the result as a string. If D is 0, the result has no decimal point or fractional part. | ||||||||||||||
string | get_json_object(string json_string, string path) | Extracts json object from a json string based on json path specified, and returns json string of the extracted json object. It will return null if the input json string is invalid.
| ||||||||||||||
boolean | in_file(string str, string filename) | Returns true if the string str appears as an entire line in filename. | ||||||||||||||
int | instr(string str, string substr) | Returns the position of the first occurrence of | ||||||||||||||
int | length(string A) | Returns the length of the string. | ||||||||||||||
int | locate(string substr, string str[, int pos]) | Returns the position of the first occurrence of substr in str after position pos. | ||||||||||||||
string | lower(string A) lcase(string A) | Returns the string resulting from converting all characters of B to lower case. For example, lower('fOoBaR') results in 'foobar'. | ||||||||||||||
string | lpad(string str, int len, string pad) | Returns str, left-padded with pad to a length of len. If str is longer than len, the return value is shortened to len characters. In case of empty pad string, the return value is null. | ||||||||||||||
string | ltrim(string A) | Returns the string resulting from trimming spaces from the beginning(left hand side) of A. For example, ltrim(' foobar ') results in 'foobar '. | ||||||||||||||
array<struct<string,double>> | ngrams(array<array<string>>, int N, int K, int pf) | Returns the top-k N-grams from a set of tokenized sentences, such as those returned by the sentences() UDAF. See StatisticsAndDataMining for more information. | ||||||||||||||
int | octet_length(string str) | Returns the number of octets required to hold the string str in UTF-8 encoding. Note that octet_length(str) can be larger than character_length(str). | ||||||||||||||
string | parse_url(string urlString, string partToExtract [, string keyToExtract]) | Returns the specified part from the URL. Valid values for partToExtract include HOST, PATH, QUERY, REF, PROTOCOL, AUTHORITY, FILE, and USERINFO. For example, parse_url('http://facebook.com/path1/p.php?k1=v1&k2=v2#Ref1', 'HOST') returns 'facebook.com'. Also, a value of a particular key in QUERY can be extracted by providing the key as the third argument, for example, parse_url('http://facebook.com/path1/p.php?k1=v1&k2=v2#Ref1', 'QUERY', 'k1') returns 'v1'. | ||||||||||||||
string | printf(String format, Obj... args) | Returns the input formatted according to printf-style format strings. | ||||||||||||||
string | quote(String text) | Returns the quoted string
| ||||||||||||||
string | regexp_extract(string subject, string pattern, int index) | Returns the string extracted using the pattern. For example, regexp_extract('foothebar', 'foo(.*?)(bar)', 2) returns 'bar.' Note that some care is necessary in using predefined character classes: using '\s' as the second argument will match the letter s; '\\s' is necessary to match whitespace, etc. The 'index' parameter is the Java regex Matcher group() method index. | ||||||||||||||
string | regexp_replace(string INITIAL_STRING, string PATTERN, string REPLACEMENT) | Returns the string resulting from replacing all substrings in INITIAL_STRING that match the java regular expression syntax defined in PATTERN with instances of REPLACEMENT. For example, regexp_replace("foobar", "oo|ar", "") returns 'fb.' Note that some care is necessary in using predefined character classes: using '\s' as the second argument will match the letter s; '\\s' is necessary to match whitespace, etc. | ||||||||||||||
string | repeat(string str, int n) | Repeats str n times. | ||||||||||||||
string | replace(string A, string OLD, string NEW) | Returns the string A with all non-overlapping occurrences of OLD replaced with NEW. Example: select replace("ababab", "abab", "Z"); returns "Zab". | ||||||||||||||
string | reverse(string A) | Returns the reversed string. | ||||||||||||||
string | rpad(string str, int len, string pad) | Returns str, right-padded with pad to a length of len. If str is longer than len, the return value is shortened to len characters. In case of empty pad string, the return value is null. | ||||||||||||||
string | rtrim(string A) | Returns the string resulting from trimming spaces from the end(right hand side) of A. For example, rtrim(' foobar ') results in ' foobar'. | ||||||||||||||
array<array<string>> | sentences(string str, string lang, string locale) | Tokenizes a string of natural language text into words and sentences, where each sentence is broken at the appropriate sentence boundary and returned as an array of words. The 'lang' and 'locale' are optional arguments. For example, sentences('Hello there! How are you?') returns ( ("Hello", "there"), ("How", "are", "you") ). | ||||||||||||||
string | space(int n) | Returns a string of n spaces. | ||||||||||||||
array | split(string str, string pat) | Splits str around pat (pat is a regular expression). | ||||||||||||||
map<string,string> | str_to_map(text[, delimiter1, delimiter2]) | Splits text into key-value pairs using two delimiters. Delimiter1 separates text into K-V pairs, and Delimiter2 splits each K-V pair. Default delimiters are ',' for delimiter1 and ':' for delimiter2. | ||||||||||||||
string | substr(string|binary A, int start) substring(string|binary A, int start) | Returns the substring or slice of the byte array of A starting from start position till the end of string A. For example, substr('foobar', 4) results in 'bar'. | ||||||||||||||
string | substr(string|binary A, int start, int len) substring(string|binary A, int start, int len) | Returns the substring or slice of the byte array of A starting from start position with length len. For example, substr('foobar', 4, 1) results in 'b'. | ||||||||||||||
string | substring_index(string A, string delim, int count) | Returns the substring from string A before count occurrences of the delimiter delim. If count is positive, everything to the left of the final delimiter (counting from the left) is returned. If count is negative, everything to the right of the final delimiter (counting from the right) is returned. Substring_index performs a case-sensitive match when searching for delim. Example: substring_index('www.apache.org', '.', 2) = 'www.apache'. | ||||||||||||||
string | translate(string|char|varchar input, string|char|varchar from, string|char|varchar to) | Translates the input string by replacing the characters present in the | ||||||||||||||
string | trim(string A) | Returns the string resulting from trimming spaces from both ends of A. For example, trim(' foobar ') results in 'foobar' | ||||||||||||||
binary | unbase64(string str) | Converts the argument from a base 64 string to BINARY. | ||||||||||||||
string | upper(string A) ucase(string A) | Returns the string resulting from converting all characters of A to upper case. For example, upper('fOoBaR') results in 'FOOBAR'. | ||||||||||||||
string | initcap(string A) | Returns string, with the first letter of each word in uppercase, all other letters in lowercase. Words are delimited by whitespace. | ||||||||||||||
int | levenshtein(string A, string B) | Returns the Levenshtein distance between two strings. For example, Example: levenshtein('kitten', 'sitting') results in 3. | ||||||||||||||
string | soundex(string A) | Returns the soundex code of the string. For example, Example: soundex('Miller') results in M460. |
Type Conversion Functions
The following built-in type conversion functions are supported in Hive.
Return Type | Name(Signature) | Description |
---|---|---|
binary | binary(string|binary) | Casts the parameter into a binary. |
Expected "=" to follow "type" | cast(expr as <type>) | Converts the results of the expression expr to <type>. For example, cast('1' as BIGINT) will convert the string '1' to its integral representation. A null is returned if the conversion does not succeed. If cast(expr as boolean) Hive returns true for a non-empty string. |
Conditional Functions
The following built-in conditional functions are supported in Hive.
Return Type | Name(Signature) | Description | |||||
---|---|---|---|---|---|---|---|
T | if(boolean testCondition, T valueTrue, T valueFalseOrNull) | Returns valueTrue when testCondition is true, returns valueFalseOrNull otherwise. | |||||
boolean | isnull( a ) | Returns true if a is NULL and false otherwise. | |||||
boolean | isnotnull ( a ) | Returns true if a is not NULL and false otherwise. | |||||
T | nvl(T value, T default_value) | Returns default value if value is null else returns value. | |||||
T | COALESCE(T v1, T v2, ...) | Returns the first v that is not NULL, or NULL if all v's are NULL. | |||||
T | CASE a WHEN b THEN c [WHEN d THEN e]* [ELSE f] END | When a = b, returns c; when a = d, returns e; else returns f. | |||||
T | CASE WHEN a THEN b [WHEN c THEN d]* [ELSE e] END | When a = true, returns b; when c = true, returns d; else returns e. | |||||
T | nullif( a, b ) | Returns NULL if a=b; otherwise returns a.
| |||||
void | assert_true(boolean condition) | Throw an exception if 'condition' is not true, otherwise return null. For example, select assert_true (2<1). |
Data Masking Functions
Misc. Functions
Geospatial Status colour Green title New
colour | Green |
---|---|
title | New |
Built-in Aggregate Functions (UDAF)
Built-in Table-Generating Functions (UDTF)
Creating Custom UDF's
Apache Hive has rich built-in UDFs, but still, if you need something special that the built-in functions did not cover you can write your own. Just a small Java knowledge needs to write a custom UDF.
UDFs can be divided into three types depending on the number of input rows and the number of output rows returned. Each of these UDFs needs to derive (implement) a different interface.
UDF | UDAF | UDTF |
---|---|---|
It is a function that receives only a single row as an input and returns a single row as an output. Like: length, or round functions Extend class: UDF | It is a function that receives multiple rows as input and returns a single row as output. Like: Count, Min, Max | It is a function that receives a single row as input and returns multiple rows - result set or table - as output. |