functions

java.lang.Object
- org.apache.spark.sql.functions

```
public class functions
extends java.lang.Object
```
:: Experimental :: Functions available for DataFrame.

Since:

1.3.0

Constructor Summary

Constructors
Constructor and Description

functions()

Constructors
Constructor and Description
`functions()`

Method Summary

Methods
Modifier and Type	Method and Description
`static Column`	`abs(Column e)` Computes the absolute value.
`static Column`	`acos(Column e)` Computes the cosine inverse of the given value; the returned angle is in the range 0.0 through pi.
`static Column`	`acos(java.lang.String columnName)` Computes the cosine inverse of the given column; the returned angle is in the range 0.0 through pi.
`static Column`	`add_months(Column startDate, int numMonths)` Returns the date that is numMonths after startDate.
`static Column`	`approxCountDistinct(Column e)` Aggregate function: returns the approximate number of distinct items in a group.
`static Column`	`approxCountDistinct(Column e, double rsd)` Aggregate function: returns the approximate number of distinct items in a group.
`static Column`	`approxCountDistinct(java.lang.String columnName)` Aggregate function: returns the approximate number of distinct items in a group.
`static Column`	`approxCountDistinct(java.lang.String columnName, double rsd)` Aggregate function: returns the approximate number of distinct items in a group.
`static Column`	`array_contains(Column column, java.lang.Object value)` Returns true if the array contains `value`
`static Column`	`array(Column... cols)` Creates a new array column.
`static Column`	`array(scala.collection.Seq<Column> cols)` Creates a new array column.
`static Column`	`array(java.lang.String colName, scala.collection.Seq<java.lang.String> colNames)` Creates a new array column.
`static Column`	`array(java.lang.String colName, java.lang.String... colNames)` Creates a new array column.
`static Column`	`asc(java.lang.String columnName)` Returns a sort expression based on ascending order of the column.
`static Column`	`ascii(Column e)` Computes the numeric value of the first character of the string column, and returns the result as a int column.
`static Column`	`asin(Column e)` Computes the sine inverse of the given value; the returned angle is in the range -pi/2 through pi/2.
`static Column`	`asin(java.lang.String columnName)` Computes the sine inverse of the given column; the returned angle is in the range -pi/2 through pi/2.
`static Column`	`atan(Column e)` Computes the tangent inverse of the given value.
`static Column`	`atan(java.lang.String columnName)` Computes the tangent inverse of the given column.
`static Column`	`atan2(Column l, Column r)` Returns the angle theta from the conversion of rectangular coordinates (x, y) to polar coordinates (r, theta).
`static Column`	`atan2(Column l, double r)` Returns the angle theta from the conversion of rectangular coordinates (x, y) to polar coordinates (r, theta).
`static Column`	`atan2(Column l, java.lang.String rightName)` Returns the angle theta from the conversion of rectangular coordinates (x, y) to polar coordinates (r, theta).
`static Column`	`atan2(double l, Column r)` Returns the angle theta from the conversion of rectangular coordinates (x, y) to polar coordinates (r, theta).
`static Column`	`atan2(double l, java.lang.String rightName)` Returns the angle theta from the conversion of rectangular coordinates (x, y) to polar coordinates (r, theta).
`static Column`	`atan2(java.lang.String leftName, Column r)` Returns the angle theta from the conversion of rectangular coordinates (x, y) to polar coordinates (r, theta).
`static Column`	`atan2(java.lang.String leftName, double r)` Returns the angle theta from the conversion of rectangular coordinates (x, y) to polar coordinates (r, theta).
`static Column`	`atan2(java.lang.String leftName, java.lang.String rightName)` Returns the angle theta from the conversion of rectangular coordinates (x, y) to polar coordinates (r, theta).
`static Column`	`avg(Column e)` Aggregate function: returns the average of the values in a group.
`static Column`	`avg(java.lang.String columnName)` Aggregate function: returns the average of the values in a group.
`static Column`	`base64(Column e)` Computes the BASE64 encoding of a binary column and returns it as a string column.
`static Column`	`bin(Column e)` An expression that returns the string representation of the binary value of the given long column.
`static Column`	`bin(java.lang.String columnName)` An expression that returns the string representation of the binary value of the given long column.
`static Column`	`bitwiseNOT(Column e)` Computes bitwise NOT.
`static <T> Dataset<T>`	`broadcast(Dataset<T> df)` Marks a DataFrame as small enough for use in broadcast joins.
`static Column`	`bround(Column e)` Returns the value of the column `e` rounded to 0 decimal places with HALF_EVEN round mode.
`static Column`	`bround(Column e, int scale)` Round the value of `e` to `scale` decimal places with HALF_EVEN round mode if `scale` >= 0 or at integral part when `scale` < 0.
`static Column`	`callUDF(java.lang.String udfName, Column... cols)` Call an user-defined function.
`static Column`	`callUDF(java.lang.String udfName, scala.collection.Seq<Column> cols)` Call an user-defined function.
`static Column`	`cbrt(Column e)` Computes the cube-root of the given value.
`static Column`	`cbrt(java.lang.String columnName)` Computes the cube-root of the given column.
`static Column`	`ceil(Column e)` Computes the ceiling of the given value.
`static Column`	`ceil(java.lang.String columnName)` Computes the ceiling of the given column.
`static Column`	`coalesce(Column... e)` Returns the first column that is not null, or null if all inputs are null.
`static Column`	`coalesce(scala.collection.Seq<Column> e)` Returns the first column that is not null, or null if all inputs are null.
`static Column`	`col(java.lang.String colName)` Returns a `Column` based on the given column name.
`static Column`	`collect_list(Column e)` Aggregate function: returns a list of objects with duplicates.
`static Column`	`collect_list(java.lang.String columnName)` Aggregate function: returns a list of objects with duplicates.
`static Column`	`collect_set(Column e)` Aggregate function: returns a set of objects with duplicate elements eliminated.
`static Column`	`collect_set(java.lang.String columnName)` Aggregate function: returns a set of objects with duplicate elements eliminated.
`static Column`	`column(java.lang.String colName)` Returns a `Column` based on the given column name.
`static Column`	`concat_ws(java.lang.String sep, Column... exprs)` Concatenates multiple input string columns together into a single string column, using the given separator.
`static Column`	`concat_ws(java.lang.String sep, scala.collection.Seq<Column> exprs)` Concatenates multiple input string columns together into a single string column, using the given separator.
`static Column`	`concat(Column... exprs)` Concatenates multiple input string columns together into a single string column.
`static Column`	`concat(scala.collection.Seq<Column> exprs)` Concatenates multiple input string columns together into a single string column.
`static Column`	`conv(Column num, int fromBase, int toBase)` Convert a number in a string column from one base to another.
`static Column`	`corr(Column column1, Column column2)` Aggregate function: returns the Pearson Correlation Coefficient for two columns.
`static Column`	`corr(java.lang.String columnName1, java.lang.String columnName2)` Aggregate function: returns the Pearson Correlation Coefficient for two columns.
`static Column`	`cos(Column e)` Computes the cosine of the given value.
`static Column`	`cos(java.lang.String columnName)` Computes the cosine of the given column.
`static Column`	`cosh(Column e)` Computes the hyperbolic cosine of the given value.
`static Column`	`cosh(java.lang.String columnName)` Computes the hyperbolic cosine of the given column.
`static Column`	`count(Column e)` Aggregate function: returns the number of items in a group.
`static TypedColumn<java.lang.Object,java.lang.Object>`	`count(java.lang.String columnName)` Aggregate function: returns the number of items in a group.
`static Column`	`countDistinct(Column expr, Column... exprs)` Aggregate function: returns the number of distinct items in a group.
`static Column`	`countDistinct(Column expr, scala.collection.Seq<Column> exprs)` Aggregate function: returns the number of distinct items in a group.
`static Column`	`countDistinct(java.lang.String columnName, scala.collection.Seq<java.lang.String> columnNames)` Aggregate function: returns the number of distinct items in a group.
`static Column`	`countDistinct(java.lang.String columnName, java.lang.String... columnNames)` Aggregate function: returns the number of distinct items in a group.
`static Column`	`covar_pop(Column column1, Column column2)` Aggregate function: returns the population covariance for two columns.
`static Column`	`covar_pop(java.lang.String columnName1, java.lang.String columnName2)` Aggregate function: returns the population covariance for two columns.
`static Column`	`covar_samp(Column column1, Column column2)` Aggregate function: returns the sample covariance for two columns.
`static Column`	`covar_samp(java.lang.String columnName1, java.lang.String columnName2)` Aggregate function: returns the sample covariance for two columns.
`static Column`	`crc32(Column e)` Calculates the cyclic redundancy check value (CRC32) of a binary column and returns the value as a bigint.
`static Column`	`cume_dist()` Window function: returns the cumulative distribution of values within a window partition, i.e.
`static Column`	`current_date()` Returns the current date as a date column.
`static Column`	`current_timestamp()` Returns the current timestamp as a timestamp column.
`static Column`	`date_add(Column start, int days)` Returns the date that is `days` days after `start`
`static Column`	`date_format(Column dateExpr, java.lang.String format)` Converts a date/timestamp/string to a value of string in the format specified by the date format given by the second argument.
`static Column`	`date_sub(Column start, int days)` Returns the date that is `days` days before `start`
`static Column`	`datediff(Column end, Column start)` Returns the number of days from `start` to `end`.
`static Column`	`dayofmonth(Column e)` Extracts the day of the month as an integer from a given date/timestamp/string.
`static Column`	`dayofyear(Column e)` Extracts the day of the year as an integer from a given date/timestamp/string.
`static Column`	`decode(Column value, java.lang.String charset)` Computes the first argument into a string from a binary using the provided character set (one of 'US-ASCII', 'ISO-8859-1', 'UTF-8', 'UTF-16BE', 'UTF-16LE', 'UTF-16').
`static Column`	`dense_rank()` Window function: returns the rank of rows within a window partition, without any gaps.
`static Column`	`desc(java.lang.String columnName)` Returns a sort expression based on the descending order of the column.
`static Column`	`encode(Column value, java.lang.String charset)` Computes the first argument into a binary from a string using the provided character set (one of 'US-ASCII', 'ISO-8859-1', 'UTF-8', 'UTF-16BE', 'UTF-16LE', 'UTF-16').
`static Column`	`exp(Column e)` Computes the exponential of the given value.
`static Column`	`exp(java.lang.String columnName)` Computes the exponential of the given column.
`static Column`	`explode(Column e)` Creates a new row for each element in the given array or map column.
`static Column`	`expm1(Column e)` Computes the exponential of the given value minus one.
`static Column`	`expm1(java.lang.String columnName)` Computes the exponential of the given column.
`static Column`	`expr(java.lang.String expr)` Parses the expression string into the column that it represents, similar to DataFrame.selectExpr
`static Column`	`factorial(Column e)` Computes the factorial of the given value.
`static Column`	`first(Column e)` Aggregate function: returns the first value in a group.
`static Column`	`first(Column e, boolean ignoreNulls)` Aggregate function: returns the first value in a group.
`static Column`	`first(java.lang.String columnName)` Aggregate function: returns the first value of a column in a group.
`static Column`	`first(java.lang.String columnName, boolean ignoreNulls)` Aggregate function: returns the first value of a column in a group.
`static Column`	`floor(Column e)` Computes the floor of the given value.
`static Column`	`floor(java.lang.String columnName)` Computes the floor of the given column.
`static Column`	`format_number(Column x, int d)` Formats numeric column x to a format like '#,###,###.##', rounded to d decimal places, and returns the result as a string column.
`static Column`	`format_string(java.lang.String format, Column... arguments)` Formats the arguments in printf-style and returns the result as a string column.
`static Column`	`format_string(java.lang.String format, scala.collection.Seq<Column> arguments)` Formats the arguments in printf-style and returns the result as a string column.
`static Column`	`from_unixtime(Column ut)` Converts the number of seconds from unix epoch (1970-01-01 00:00:00 UTC) to a string representing the timestamp of that moment in the current system time zone in the given format.
`static Column`	`from_unixtime(Column ut, java.lang.String f)` Converts the number of seconds from unix epoch (1970-01-01 00:00:00 UTC) to a string representing the timestamp of that moment in the current system time zone in the given format.
`static Column`	`from_utc_timestamp(Column ts, java.lang.String tz)` Assumes given timestamp is UTC and converts to given timezone.
`static Column`	`get_json_object(Column e, java.lang.String path)` Extracts json object from a json string based on json path specified, and returns json string of the extracted json object.
`static Column`	`greatest(Column... exprs)` Returns the greatest value of the list of values, skipping null values.
`static Column`	`greatest(scala.collection.Seq<Column> exprs)` Returns the greatest value of the list of values, skipping null values.
`static Column`	`greatest(java.lang.String columnName, scala.collection.Seq<java.lang.String> columnNames)` Returns the greatest value of the list of column names, skipping null values.
`static Column`	`greatest(java.lang.String columnName, java.lang.String... columnNames)` Returns the greatest value of the list of column names, skipping null values.
`static Column`	`grouping_id(scala.collection.Seq<Column> cols)` Aggregate function: returns the level of grouping, equals to
`static Column`	`grouping_id(java.lang.String colName, scala.collection.Seq<java.lang.String> colNames)` Aggregate function: returns the level of grouping, equals to
`static Column`	`grouping(Column e)` Aggregate function: indicates whether a specified column in a GROUP BY list is aggregated or not, returns 1 for aggregated or 0 for not aggregated in the result set.
`static Column`	`grouping(java.lang.String columnName)` Aggregate function: indicates whether a specified column in a GROUP BY list is aggregated or not, returns 1 for aggregated or 0 for not aggregated in the result set.
`static Column`	`hash(Column... cols)` Calculates the hash code of given columns, and returns the result as an int column.
`static Column`	`hash(scala.collection.Seq<Column> cols)` Calculates the hash code of given columns, and returns the result as an int column.
`static Column`	`hex(Column column)` Computes hex value of the given column.
`static Column`	`hour(Column e)` Extracts the hours as an integer from a given date/timestamp/string.
`static Column`	`hypot(Column l, Column r)` Computes `sqrt(a^2^ + b^2^)` without intermediate overflow or underflow.
`static Column`	`hypot(Column l, double r)` Computes `sqrt(a^2^ + b^2^)` without intermediate overflow or underflow.
`static Column`	`hypot(Column l, java.lang.String rightName)` Computes `sqrt(a^2^ + b^2^)` without intermediate overflow or underflow.
`static Column`	`hypot(double l, Column r)` Computes `sqrt(a^2^ + b^2^)` without intermediate overflow or underflow.
`static Column`	`hypot(double l, java.lang.String rightName)` Computes `sqrt(a^2^ + b^2^)` without intermediate overflow or underflow.
`static Column`	`hypot(java.lang.String leftName, Column r)` Computes `sqrt(a^2^ + b^2^)` without intermediate overflow or underflow.
`static Column`	`hypot(java.lang.String leftName, double r)` Computes `sqrt(a^2^ + b^2^)` without intermediate overflow or underflow.
`static Column`	`hypot(java.lang.String leftName, java.lang.String rightName)` Computes `sqrt(a^2^ + b^2^)` without intermediate overflow or underflow.
`static Column`	`initcap(Column e)` Returns a new string column by converting the first letter of each word to uppercase.
`static Column`	`input_file_name()` Creates a string column for the file name of the current Spark task.
`static Column`	`instr(Column str, java.lang.String substring)` Locate the position of the first occurrence of substr column in the given string.
`static Column`	`isnan(Column e)` Return true iff the column is NaN.
`static Column`	`isnull(Column e)` Return true iff the column is null.
`static Column`	`json_tuple(Column json, scala.collection.Seq<java.lang.String> fields)` Creates a new row for a json column according to the given field names.
`static Column`	`json_tuple(Column json, java.lang.String... fields)` Creates a new row for a json column according to the given field names.
`static Column`	`kurtosis(Column e)` Aggregate function: returns the kurtosis of the values in a group.
`static Column`	`kurtosis(java.lang.String columnName)` Aggregate function: returns the kurtosis of the values in a group.
`static Column`	`lag(Column e, int offset)` Window function: returns the value that is `offset` rows before the current row, and `null` if there is less than `offset` rows before the current row.
`static Column`	`lag(Column e, int offset, java.lang.Object defaultValue)` Window function: returns the value that is `offset` rows before the current row, and `defaultValue` if there is less than `offset` rows before the current row.
`static Column`	`lag(java.lang.String columnName, int offset)` Window function: returns the value that is `offset` rows before the current row, and `null` if there is less than `offset` rows before the current row.
`static Column`	`lag(java.lang.String columnName, int offset, java.lang.Object defaultValue)` Window function: returns the value that is `offset` rows before the current row, and `defaultValue` if there is less than `offset` rows before the current row.
`static Column`	`last_day(Column e)` Given a date column, returns the last day of the month which the given date belongs to.
`static Column`	`last(Column e)` Aggregate function: returns the last value in a group.
`static Column`	`last(Column e, boolean ignoreNulls)` Aggregate function: returns the last value in a group.
`static Column`	`last(java.lang.String columnName)` Aggregate function: returns the last value of the column in a group.
`static Column`	`last(java.lang.String columnName, boolean ignoreNulls)` Aggregate function: returns the last value of the column in a group.
`static Column`	`lead(Column e, int offset)` Window function: returns the value that is `offset` rows after the current row, and `null` if there is less than `offset` rows after the current row.
`static Column`	`lead(Column e, int offset, java.lang.Object defaultValue)` Window function: returns the value that is `offset` rows after the current row, and `defaultValue` if there is less than `offset` rows after the current row.
`static Column`	`lead(java.lang.String columnName, int offset)` Window function: returns the value that is `offset` rows after the current row, and `null` if there is less than `offset` rows after the current row.
`static Column`	`lead(java.lang.String columnName, int offset, java.lang.Object defaultValue)` Window function: returns the value that is `offset` rows after the current row, and `defaultValue` if there is less than `offset` rows after the current row.
`static Column`	`least(Column... exprs)` Returns the least value of the list of values, skipping null values.
`static Column`	`least(scala.collection.Seq<Column> exprs)` Returns the least value of the list of values, skipping null values.
`static Column`	`least(java.lang.String columnName, scala.collection.Seq<java.lang.String> columnNames)` Returns the least value of the list of column names, skipping null values.
`static Column`	`least(java.lang.String columnName, java.lang.String... columnNames)` Returns the least value of the list of column names, skipping null values.
`static Column`	`length(Column e)` Computes the length of a given string or binary column.
`static Column`	`levenshtein(Column l, Column r)` Computes the Levenshtein distance of the two given string columns.
`static Column`	`lit(java.lang.Object literal)` Creates a `Column` of literal value.
`static Column`	`locate(java.lang.String substr, Column str)` Locate the position of the first occurrence of substr.
`static Column`	`locate(java.lang.String substr, Column str, int pos)` Locate the position of the first occurrence of substr in a string column, after position pos.
`static Column`	`log(Column e)` Computes the natural logarithm of the given value.
`static Column`	`log(double base, Column a)` Returns the first argument-base logarithm of the second argument.
`static Column`	`log(double base, java.lang.String columnName)` Returns the first argument-base logarithm of the second argument.
`static Column`	`log(java.lang.String columnName)` Computes the natural logarithm of the given column.
`static Column`	`log10(Column e)` Computes the logarithm of the given value in base 10.
`static Column`	`log10(java.lang.String columnName)` Computes the logarithm of the given value in base 10.
`static Column`	`log1p(Column e)` Computes the natural logarithm of the given value plus one.
`static Column`	`log1p(java.lang.String columnName)` Computes the natural logarithm of the given column plus one.
`static Column`	`log2(Column expr)` Computes the logarithm of the given column in base 2.
`static Column`	`log2(java.lang.String columnName)` Computes the logarithm of the given value in base 2.
`static Column`	`lower(Column e)` Converts a string column to lower case.
`static Column`	`lpad(Column str, int len, java.lang.String pad)` Left-pad the string column with
`static Column`	`ltrim(Column e)` Trim the spaces from left end for the specified string value.
`static Column`	`map(Column... cols)` Creates a new map column.
`static Column`	`map(scala.collection.Seq<Column> cols)` Creates a new map column.
`static Column`	`max(Column e)` Aggregate function: returns the maximum value of the expression in a group.
`static Column`	`max(java.lang.String columnName)` Aggregate function: returns the maximum value of the column in a group.
`static Column`	`md5(Column e)` Calculates the MD5 digest of a binary column and returns the value as a 32 character hex string.
`static Column`	`mean(Column e)` Aggregate function: returns the average of the values in a group.
`static Column`	`mean(java.lang.String columnName)` Aggregate function: returns the average of the values in a group.
`static Column`	`min(Column e)` Aggregate function: returns the minimum value of the expression in a group.
`static Column`	`min(java.lang.String columnName)` Aggregate function: returns the minimum value of the column in a group.
`static Column`	`minute(Column e)` Extracts the minutes as an integer from a given date/timestamp/string.
`static Column`	`monotonically_increasing_id()` A column expression that generates monotonically increasing 64-bit integers.
`static Column`	`monotonicallyIncreasingId()` A column expression that generates monotonically increasing 64-bit integers.
`static Column`	`month(Column e)` Extracts the month as an integer from a given date/timestamp/string.
`static Column`	`months_between(Column date1, Column date2)`
`static Column`	`nanvl(Column col1, Column col2)` Returns col1 if it is not NaN, or col2 if col1 is NaN.
`static Column`	`negate(Column e)` Unary minus, i.e.
`static Column`	`next_day(Column date, java.lang.String dayOfWeek)` Given a date column, returns the first date which is later than the value of the date column that is on the specified day of the week.
`static Column`	`not(Column e)` Inversion of boolean expression, i.e.
`static Column`	`ntile(int n)` Window function: returns the ntile group id (from 1 to `n` inclusive) in an ordered window partition.
`static Column`	`percent_rank()` Window function: returns the relative rank (i.e.
`static Column`	`pmod(Column dividend, Column divisor)` Returns the positive value of dividend mod divisor.
`static Column`	`pow(Column l, Column r)` Returns the value of the first argument raised to the power of the second argument.
`static Column`	`pow(Column l, double r)` Returns the value of the first argument raised to the power of the second argument.
`static Column`	`pow(Column l, java.lang.String rightName)` Returns the value of the first argument raised to the power of the second argument.
`static Column`	`pow(double l, Column r)` Returns the value of the first argument raised to the power of the second argument.
`static Column`	`pow(double l, java.lang.String rightName)` Returns the value of the first argument raised to the power of the second argument.
`static Column`	`pow(java.lang.String leftName, Column r)` Returns the value of the first argument raised to the power of the second argument.
`static Column`	`pow(java.lang.String leftName, double r)` Returns the value of the first argument raised to the power of the second argument.
`static Column`	`pow(java.lang.String leftName, java.lang.String rightName)` Returns the value of the first argument raised to the power of the second argument.
`static Column`	`quarter(Column e)` Extracts the quarter as an integer from a given date/timestamp/string.
`static Column`	`rand()` Generate a random column with i.i.d.
`static Column`	`rand(long seed)` Generate a random column with i.i.d.
`static Column`	`randn()` Generate a column with i.i.d.
`static Column`	`randn(long seed)` Generate a column with i.i.d.
`static Column`	`rank()` Window function: returns the rank of rows within a window partition.
`static Column`	`regexp_extract(Column e, java.lang.String exp, int groupIdx)` Extract a specific(idx) group identified by a java regex, from the specified string column.
`static Column`	`regexp_replace(Column e, java.lang.String pattern, java.lang.String replacement)` Replace all substrings of the specified string value that match regexp with rep.
`static Column`	`repeat(Column str, int n)` Repeats a string column n times, and returns it as a new string column.
`static Column`	`reverse(Column str)` Reverses the string column and returns it as a new string column.
`static Column`	`rint(Column e)` Returns the double value that is closest in value to the argument and is equal to a mathematical integer.
`static Column`	`rint(java.lang.String columnName)` Returns the double value that is closest in value to the argument and is equal to a mathematical integer.
`static Column`	`round(Column e)` Returns the value of the column `e` rounded to 0 decimal places.
`static Column`	`round(Column e, int scale)` Round the value of `e` to `scale` decimal places if `scale` >= 0 or at integral part when `scale` < 0.
`static Column`	`row_number()` Window function: returns a sequential number starting at 1 within a window partition.
`static Column`	`rpad(Column str, int len, java.lang.String pad)` Right-padded with pad to a length of len.
`static Column`	`rtrim(Column e)` Trim the spaces from right end for the specified string value.
`static Column`	`second(Column e)` Extracts the seconds as an integer from a given date/timestamp/string.
`static Column`	`sha1(Column e)` Calculates the SHA-1 digest of a binary column and returns the value as a 40 character hex string.
`static Column`	`sha2(Column e, int numBits)` Calculates the SHA-2 family of hash functions of a binary column and returns the value as a hex string.
`static Column`	`shiftLeft(Column e, int numBits)` Shift the given value numBits left.
`static Column`	`shiftRight(Column e, int numBits)` Shift the given value numBits right.
`static Column`	`shiftRightUnsigned(Column e, int numBits)` Unsigned shift the given value numBits right.
`static Column`	`signum(Column e)` Computes the signum of the given value.
`static Column`	`signum(java.lang.String columnName)` Computes the signum of the given column.
`static Column`	`sin(Column e)` Computes the sine of the given value.
`static Column`	`sin(java.lang.String columnName)` Computes the sine of the given column.
`static Column`	`sinh(Column e)` Computes the hyperbolic sine of the given value.
`static Column`	`sinh(java.lang.String columnName)` Computes the hyperbolic sine of the given column.
`static Column`	`size(Column e)` Returns length of array or map.
`static Column`	`skewness(Column e)` Aggregate function: returns the skewness of the values in a group.
`static Column`	`skewness(java.lang.String columnName)` Aggregate function: returns the skewness of the values in a group.
`static Column`	`sort_array(Column e)` Sorts the input array for the given column in ascending order, according to the natural ordering of the array elements.
`static Column`	`sort_array(Column e, boolean asc)` Sorts the input array for the given column in ascending / descending order, according to the natural ordering of the array elements.
`static Column`	`soundex(Column e)` * Return the soundex code for the specified expression.
`static Column`	`spark_partition_id()` Partition ID of the Spark task.
`static Column`	`split(Column str, java.lang.String pattern)` Splits str around pattern (pattern is a regular expression).
`static Column`	`sqrt(Column e)` Computes the square root of the specified float value.
`static Column`	`sqrt(java.lang.String colName)` Computes the square root of the specified float value.
`static Column`	`stddev_pop(Column e)` Aggregate function: returns the population standard deviation of the expression in a group.
`static Column`	`stddev_pop(java.lang.String columnName)` Aggregate function: returns the population standard deviation of the expression in a group.
`static Column`	`stddev_samp(Column e)` Aggregate function: returns the sample standard deviation of the expression in a group.
`static Column`	`stddev_samp(java.lang.String columnName)` Aggregate function: returns the sample standard deviation of the expression in a group.
`static Column`	`stddev(Column e)` Aggregate function: alias for `stddev_samp`.
`static Column`	`stddev(java.lang.String columnName)` Aggregate function: alias for `stddev_samp`.
`static Column`	`struct(Column... cols)` Creates a new struct column.
`static Column`	`struct(scala.collection.Seq<Column> cols)` Creates a new struct column.
`static Column`	`struct(java.lang.String colName, scala.collection.Seq<java.lang.String> colNames)` Creates a new struct column that composes multiple input columns.
`static Column`	`struct(java.lang.String colName, java.lang.String... colNames)` Creates a new struct column that composes multiple input columns.
`static Column`	`substring_index(Column str, java.lang.String delim, int count)` Returns the substring from string str before count occurrences of the delimiter delim.
`static Column`	`substring(Column str, int pos, int len)` Substring starts at `pos` and is of length `len` when str is String type or returns the slice of byte array that starts at `pos` in byte and is of length `len` when str is Binary type
`static Column`	`sum(Column e)` Aggregate function: returns the sum of all values in the expression.
`static Column`	`sum(java.lang.String columnName)` Aggregate function: returns the sum of all values in the given column.
`static Column`	`sumDistinct(Column e)` Aggregate function: returns the sum of distinct values in the expression.
`static Column`	`sumDistinct(java.lang.String columnName)` Aggregate function: returns the sum of distinct values in the expression.
`static Column`	`tan(Column e)` Computes the tangent of the given value.
`static Column`	`tan(java.lang.String columnName)` Computes the tangent of the given column.
`static Column`	`tanh(Column e)` Computes the hyperbolic tangent of the given value.
`static Column`	`tanh(java.lang.String columnName)` Computes the hyperbolic tangent of the given column.
`static Column`	`to_date(Column e)` Converts the column into DateType.
`static Column`	`to_utc_timestamp(Column ts, java.lang.String tz)` Assumes given timestamp is in given timezone and converts to UTC.
`static Column`	`toDegrees(Column e)` Converts an angle measured in radians to an approximately equivalent angle measured in degrees.
`static Column`	`toDegrees(java.lang.String columnName)` Converts an angle measured in radians to an approximately equivalent angle measured in degrees.
`static Column`	`toRadians(Column e)` Converts an angle measured in degrees to an approximately equivalent angle measured in radians.
`static Column`	`toRadians(java.lang.String columnName)` Converts an angle measured in degrees to an approximately equivalent angle measured in radians.
`static Column`	`translate(Column src, java.lang.String matchingString, java.lang.String replaceString)` Translate any character in the src by a character in replaceString.
`static Column`	`trim(Column e)` Trim the spaces from both ends for the specified string column.
`static Column`	`trunc(Column date, java.lang.String format)` Returns date truncated to the unit specified by the format.
`static <RT> UserDefinedFunction`	`udf(scala.Function0<RT> f, scala.reflect.api.TypeTags.TypeTag<RT> evidence$1)` Defines a user-defined function of 0 arguments as user-defined function (UDF).
`static <RT,A1> UserDefinedFunction`	`udf(scala.Function1<A1,RT> f, scala.reflect.api.TypeTags.TypeTag<RT> evidence$2, scala.reflect.api.TypeTags.TypeTag<A1> evidence$3)` Defines a user-defined function of 1 arguments as user-defined function (UDF).
`static <RT,A1,A2,A3,A4,A5,A6,A7,A8,A9,A10> UserDefinedFunction`	udf(scala.Function10<A1,A2,A3,A4,A5,A6,A7,A8,A9,A10,RT> f, scala.reflect.api.TypeTags.TypeTag<RT> evidence$56, scala.reflect.api.TypeTags.TypeTag<A1> evidence$57, scala.reflect.api.TypeTags.TypeTag<A2> evidence$58, scala.reflect.api.TypeTags.TypeTag<A3> evidence$59, scala.reflect.api.TypeTags.TypeTag<A4> evidence$60, scala.reflect.api.TypeTags.TypeTag<A5> evidence$61, scala.reflect.api.TypeTags.TypeTag<A6> evidence$62, scala.reflect.api.TypeTags.TypeTag<A7> evidence$63, scala.reflect.api.TypeTags.TypeTag<A8> evidence$64, scala.reflect.api.TypeTags.TypeTag<A9> evidence$65, scala.reflect.api.TypeTags.TypeTag<A10> evidence$66) Defines a user-defined function of 10 arguments as user-defined function (UDF).
`static <RT,A1,A2> UserDefinedFunction`	`udf(scala.Function2<A1,A2,RT> f, scala.reflect.api.TypeTags.TypeTag<RT> evidence$4, scala.reflect.api.TypeTags.TypeTag<A1> evidence$5, scala.reflect.api.TypeTags.TypeTag<A2> evidence$6)` Defines a user-defined function of 2 arguments as user-defined function (UDF).
`static <RT,A1,A2,A3> UserDefinedFunction`	`udf(scala.Function3<A1,A2,A3,RT> f, scala.reflect.api.TypeTags.TypeTag<RT> evidence$7, scala.reflect.api.TypeTags.TypeTag<A1> evidence$8, scala.reflect.api.TypeTags.TypeTag<A2> evidence$9, scala.reflect.api.TypeTags.TypeTag<A3> evidence$10)` Defines a user-defined function of 3 arguments as user-defined function (UDF).
`static <RT,A1,A2,A3,A4> UserDefinedFunction`	`udf(scala.Function4<A1,A2,A3,A4,RT> f, scala.reflect.api.TypeTags.TypeTag<RT> evidence$11, scala.reflect.api.TypeTags.TypeTag<A1> evidence$12, scala.reflect.api.TypeTags.TypeTag<A2> evidence$13, scala.reflect.api.TypeTags.TypeTag<A3> evidence$14, scala.reflect.api.TypeTags.TypeTag<A4> evidence$15)` Defines a user-defined function of 4 arguments as user-defined function (UDF).
`static <RT,A1,A2,A3,A4,A5> UserDefinedFunction`	`udf(scala.Function5<A1,A2,A3,A4,A5,RT> f, scala.reflect.api.TypeTags.TypeTag<RT> evidence$16, scala.reflect.api.TypeTags.TypeTag<A1> evidence$17, scala.reflect.api.TypeTags.TypeTag<A2> evidence$18, scala.reflect.api.TypeTags.TypeTag<A3> evidence$19, scala.reflect.api.TypeTags.TypeTag<A4> evidence$20, scala.reflect.api.TypeTags.TypeTag<A5> evidence$21)` Defines a user-defined function of 5 arguments as user-defined function (UDF).
`static <RT,A1,A2,A3,A4,A5,A6> UserDefinedFunction`	`udf(scala.Function6<A1,A2,A3,A4,A5,A6,RT> f, scala.reflect.api.TypeTags.TypeTag<RT> evidence$22, scala.reflect.api.TypeTags.TypeTag<A1> evidence$23, scala.reflect.api.TypeTags.TypeTag<A2> evidence$24, scala.reflect.api.TypeTags.TypeTag<A3> evidence$25, scala.reflect.api.TypeTags.TypeTag<A4> evidence$26, scala.reflect.api.TypeTags.TypeTag<A5> evidence$27, scala.reflect.api.TypeTags.TypeTag<A6> evidence$28)` Defines a user-defined function of 6 arguments as user-defined function (UDF).
`static <RT,A1,A2,A3,A4,A5,A6,A7> UserDefinedFunction`	`udf(scala.Function7<A1,A2,A3,A4,A5,A6,A7,RT> f, scala.reflect.api.TypeTags.TypeTag<RT> evidence$29, scala.reflect.api.TypeTags.TypeTag<A1> evidence$30, scala.reflect.api.TypeTags.TypeTag<A2> evidence$31, scala.reflect.api.TypeTags.TypeTag<A3> evidence$32, scala.reflect.api.TypeTags.TypeTag<A4> evidence$33, scala.reflect.api.TypeTags.TypeTag<A5> evidence$34, scala.reflect.api.TypeTags.TypeTag<A6> evidence$35, scala.reflect.api.TypeTags.TypeTag<A7> evidence$36)` Defines a user-defined function of 7 arguments as user-defined function (UDF).
`static <RT,A1,A2,A3,A4,A5,A6,A7,A8> UserDefinedFunction`	udf(scala.Function8<A1,A2,A3,A4,A5,A6,A7,A8,RT> f, scala.reflect.api.TypeTags.TypeTag<RT> evidence$37, scala.reflect.api.TypeTags.TypeTag<A1> evidence$38, scala.reflect.api.TypeTags.TypeTag<A2> evidence$39, scala.reflect.api.TypeTags.TypeTag<A3> evidence$40, scala.reflect.api.TypeTags.TypeTag<A4> evidence$41, scala.reflect.api.TypeTags.TypeTag<A5> evidence$42, scala.reflect.api.TypeTags.TypeTag<A6> evidence$43, scala.reflect.api.TypeTags.TypeTag<A7> evidence$44, scala.reflect.api.TypeTags.TypeTag<A8> evidence$45) Defines a user-defined function of 8 arguments as user-defined function (UDF).
`static <RT,A1,A2,A3,A4,A5,A6,A7,A8,A9> UserDefinedFunction`	udf(scala.Function9<A1,A2,A3,A4,A5,A6,A7,A8,A9,RT> f, scala.reflect.api.TypeTags.TypeTag<RT> evidence$46, scala.reflect.api.TypeTags.TypeTag<A1> evidence$47, scala.reflect.api.TypeTags.TypeTag<A2> evidence$48, scala.reflect.api.TypeTags.TypeTag<A3> evidence$49, scala.reflect.api.TypeTags.TypeTag<A4> evidence$50, scala.reflect.api.TypeTags.TypeTag<A5> evidence$51, scala.reflect.api.TypeTags.TypeTag<A6> evidence$52, scala.reflect.api.TypeTags.TypeTag<A7> evidence$53, scala.reflect.api.TypeTags.TypeTag<A8> evidence$54, scala.reflect.api.TypeTags.TypeTag<A9> evidence$55) Defines a user-defined function of 9 arguments as user-defined function (UDF).
`static UserDefinedFunction`	`udf(java.lang.Object f, DataType dataType)` Defines a user-defined function (UDF) using a Scala closure.
`static Column`	`unbase64(Column e)` Decodes a BASE64 encoded string column and returns it as a binary column.
`static Column`	`unhex(Column column)` Inverse of hex.
`static Column`	`unix_timestamp()` Gets current Unix timestamp in seconds.
`static Column`	`unix_timestamp(Column s)` Converts time string in format yyyy-MM-dd HH:mm:ss to Unix timestamp (in seconds), using the default timezone and the default locale, return null if fail.
`static Column`	`unix_timestamp(Column s, java.lang.String p)` Convert time string with given pattern (see [http://docs.oracle.com/javase/tutorial/i18n/format/simpleDateFormat.html]) to Unix time stamp (in seconds), return null if fail.
`static Column`	`upper(Column e)` Converts a string column to upper case.
`static Column`	`var_pop(Column e)` Aggregate function: returns the population variance of the values in a group.
`static Column`	`var_pop(java.lang.String columnName)` Aggregate function: returns the population variance of the values in a group.
`static Column`	`var_samp(Column e)` Aggregate function: returns the unbiased variance of the values in a group.
`static Column`	`var_samp(java.lang.String columnName)` Aggregate function: returns the unbiased variance of the values in a group.
`static Column`	`variance(Column e)` Aggregate function: alias for `var_samp`.
`static Column`	`variance(java.lang.String columnName)` Aggregate function: alias for `var_samp`.
`static Column`	`weekofyear(Column e)` Extracts the week number as an integer from a given date/timestamp/string.
`static Column`	`when(Column condition, java.lang.Object value)` Evaluates a list of conditions and returns one of multiple possible result expressions.
`static Column`	`window(Column timeColumn, java.lang.String windowDuration)` Generates tumbling time windows given a timestamp specifying column.
`static Column`	`window(Column timeColumn, java.lang.String windowDuration, java.lang.String slideDuration)` Bucketize rows into one or more time windows given a timestamp specifying column.
`static Column`	`window(Column timeColumn, java.lang.String windowDuration, java.lang.String slideDuration, java.lang.String startTime)` Bucketize rows into one or more time windows given a timestamp specifying column.
`static Column`	`year(Column e)` Extracts the year as an integer from a given date/timestamp/string.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Constructor Detail
  - functions
```
public functions()
```
- Method Detail
  - countDistinct
```
public static Column countDistinct(Column expr,
                   Column... exprs)
```
    Aggregate function: returns the number of distinct items in a group.
    
    Parameters:
    expr - (undocumented)
    exprs - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - countDistinct
```
public static Column countDistinct(java.lang.String columnName,
                   java.lang.String... columnNames)
```
    Aggregate function: returns the number of distinct items in a group.
    
    Parameters:
    columnName - (undocumented)
    columnNames - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - array
```
public static Column array(Column... cols)
```
    Creates a new array column. The input columns must all have the same data type.
    
    Parameters:
    cols - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - array
```
public static Column array(java.lang.String colName,
           java.lang.String... colNames)
```
    Creates a new array column. The input columns must all have the same data type.
    
    Parameters:
    colName - (undocumented)
    colNames - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - map
```
public static Column map(Column... cols)
```
    Creates a new map column. The input columns must be grouped as key-value pairs, e.g. (key1, value1, key2, value2, ...). The key columns must all have the same data type, and can't be null. The value columns must all have the same data type.
    
    Parameters:
    cols - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    2.0
  - coalesce
```
public static Column coalesce(Column... e)
```
    Returns the first column that is not null, or null if all inputs are null.
    For example, coalesce(a, b, c) will return a if a is not null, or b if a is null and b is not null, or c if both a and b are null but c is not null.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - struct
```
public static Column struct(Column... cols)
```
    Creates a new struct column. If the input column is a column in a DataFrame, or a derived column expression that is named (i.e. aliased), its name would be remained as the StructField's name, otherwise, the newly generated StructField's name would be auto generated as col${index + 1}, i.e. col1, col2, col3, ...
    
    Parameters:
    cols - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - struct
```
public static Column struct(java.lang.String colName,
            java.lang.String... colNames)
```
    Creates a new struct column that composes multiple input columns.
    
    Parameters:
    colName - (undocumented)
    colNames - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - greatest
```
public static Column greatest(Column... exprs)
```
    Returns the greatest value of the list of values, skipping null values. This function takes at least 2 parameters. It will return null iff all parameters are null.
    
    Parameters:
    exprs - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - greatest
```
public static Column greatest(java.lang.String columnName,
              java.lang.String... columnNames)
```
    Returns the greatest value of the list of column names, skipping null values. This function takes at least 2 parameters. It will return null iff all parameters are null.
    
    Parameters:
    columnName - (undocumented)
    columnNames - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - least
```
public static Column least(Column... exprs)
```
    Returns the least value of the list of values, skipping null values. This function takes at least 2 parameters. It will return null iff all parameters are null.
    
    Parameters:
    exprs - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - least
```
public static Column least(java.lang.String columnName,
           java.lang.String... columnNames)
```
    Returns the least value of the list of column names, skipping null values. This function takes at least 2 parameters. It will return null iff all parameters are null.
    
    Parameters:
    columnName - (undocumented)
    columnNames - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - hash
```
public static Column hash(Column... cols)
```
    Calculates the hash code of given columns, and returns the result as an int column.
    
    Parameters:
    cols - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    2.0
  - concat
```
public static Column concat(Column... exprs)
```
    Concatenates multiple input string columns together into a single string column.
    
    Parameters:
    exprs - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - concat_ws
```
public static Column concat_ws(java.lang.String sep,
               Column... exprs)
```
    Concatenates multiple input string columns together into a single string column, using the given separator.
    
    Parameters:
    sep - (undocumented)
    exprs - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - format_string
```
public static Column format_string(java.lang.String format,
                   Column... arguments)
```
    Formats the arguments in printf-style and returns the result as a string column.
    
    Parameters:
    format - (undocumented)
    arguments - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - json_tuple
```
public static Column json_tuple(Column json,
                java.lang.String... fields)
```
    Creates a new row for a json column according to the given field names.
    
    Parameters:
    json - (undocumented)
    fields - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.6.0
  - callUDF
```
public static Column callUDF(java.lang.String udfName,
             Column... cols)
```
    Call an user-defined function. Example:
```
  import org.apache.spark.sql._

  val df = Seq(("id1", 1), ("id2", 4), ("id3", 5)).toDF("id", "value")
  val spark = df.sparkSession
  spark.udf.register("simpleUDF", (v: Int) => v * v)
  df.select($"id", callUDF("simpleUDF", $"value"))
 
```
    Parameters:
    udfName - (undocumented)
    cols - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - col
```
public static Column col(java.lang.String colName)
```
    Returns a Column based on the given column name.
    
    Parameters:
    colName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - column
```
public static Column column(java.lang.String colName)
```
    Returns a Column based on the given column name. Alias of col.
    
    Parameters:
    colName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - lit
```
public static Column lit(java.lang.Object literal)
```
    Creates a Column of literal value.
    The passed in object is returned directly if it is already a Column. If the object is a Scala Symbol, it is converted into a Column also. Otherwise, a new Column is created to represent the literal value.
    
    Parameters:
    literal - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - asc
```
public static Column asc(java.lang.String columnName)
```
    Returns a sort expression based on ascending order of the column.
```
   // Sort by dept in ascending order, and then age in descending order.
   df.sort(asc("dept"), desc("age"))
 
```
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - desc
```
public static Column desc(java.lang.String columnName)
```
    Returns a sort expression based on the descending order of the column.
```
   // Sort by dept in ascending order, and then age in descending order.
   df.sort(asc("dept"), desc("age"))
 
```
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - approxCountDistinct
```
public static Column approxCountDistinct(Column e)
```
    Aggregate function: returns the approximate number of distinct items in a group.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - approxCountDistinct
```
public static Column approxCountDistinct(java.lang.String columnName)
```
    Aggregate function: returns the approximate number of distinct items in a group.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - approxCountDistinct
```
public static Column approxCountDistinct(Column e,
                         double rsd)
```
    Aggregate function: returns the approximate number of distinct items in a group.
    
    Parameters:
    rsd - maximum estimation error allowed (default = 0.05)
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - approxCountDistinct
```
public static Column approxCountDistinct(java.lang.String columnName,
                         double rsd)
```
    Aggregate function: returns the approximate number of distinct items in a group.
    
    Parameters:
    rsd - maximum estimation error allowed (default = 0.05)
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - avg
```
public static Column avg(Column e)
```
    Aggregate function: returns the average of the values in a group.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - avg
```
public static Column avg(java.lang.String columnName)
```
    Aggregate function: returns the average of the values in a group.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - collect_list
```
public static Column collect_list(Column e)
```
    Aggregate function: returns a list of objects with duplicates.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.6.0
  - collect_list
```
public static Column collect_list(java.lang.String columnName)
```
    Aggregate function: returns a list of objects with duplicates.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.6.0
  - collect_set
```
public static Column collect_set(Column e)
```
    Aggregate function: returns a set of objects with duplicate elements eliminated.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.6.0
  - collect_set
```
public static Column collect_set(java.lang.String columnName)
```
    Aggregate function: returns a set of objects with duplicate elements eliminated.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.6.0
  - corr
```
public static Column corr(Column column1,
          Column column2)
```
    Aggregate function: returns the Pearson Correlation Coefficient for two columns.
    
    Parameters:
    column1 - (undocumented)
    column2 - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.6.0
  - corr
```
public static Column corr(java.lang.String columnName1,
          java.lang.String columnName2)
```
    Aggregate function: returns the Pearson Correlation Coefficient for two columns.
    
    Parameters:
    columnName1 - (undocumented)
    columnName2 - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.6.0
  - count
```
public static Column count(Column e)
```
    Aggregate function: returns the number of items in a group.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - count
```
public static TypedColumn<java.lang.Object,java.lang.Object> count(java.lang.String columnName)
```
    Aggregate function: returns the number of items in a group.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - countDistinct
```
public static Column countDistinct(Column expr,
                   scala.collection.Seq<Column> exprs)
```
    Aggregate function: returns the number of distinct items in a group.
    
    Parameters:
    expr - (undocumented)
    exprs - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - countDistinct
```
public static Column countDistinct(java.lang.String columnName,
                   scala.collection.Seq<java.lang.String> columnNames)
```
    Aggregate function: returns the number of distinct items in a group.
    
    Parameters:
    columnName - (undocumented)
    columnNames - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - covar_pop
```
public static Column covar_pop(Column column1,
               Column column2)
```
    Aggregate function: returns the population covariance for two columns.
    
    Parameters:
    column1 - (undocumented)
    column2 - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    2.0.0
  - covar_pop
```
public static Column covar_pop(java.lang.String columnName1,
               java.lang.String columnName2)
```
    Aggregate function: returns the population covariance for two columns.
    
    Parameters:
    columnName1 - (undocumented)
    columnName2 - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    2.0.0
  - covar_samp
```
public static Column covar_samp(Column column1,
                Column column2)
```
    Aggregate function: returns the sample covariance for two columns.
    
    Parameters:
    column1 - (undocumented)
    column2 - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    2.0.0
  - covar_samp
```
public static Column covar_samp(java.lang.String columnName1,
                java.lang.String columnName2)
```
    Aggregate function: returns the sample covariance for two columns.
    
    Parameters:
    columnName1 - (undocumented)
    columnName2 - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    2.0.0
  - first
```
public static Column first(Column e,
           boolean ignoreNulls)
```
    Aggregate function: returns the first value in a group.
    The function by default returns the first values it sees. It will return the first non-null value it sees when ignoreNulls is set to true. If all values are null, then null is returned.
    
    Parameters:
    e - (undocumented)
    ignoreNulls - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    2.0.0
  - first
```
public static Column first(java.lang.String columnName,
           boolean ignoreNulls)
```
    Aggregate function: returns the first value of a column in a group.
    The function by default returns the first values it sees. It will return the first non-null value it sees when ignoreNulls is set to true. If all values are null, then null is returned.
    
    Parameters:
    columnName - (undocumented)
    ignoreNulls - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    2.0.0
  - first
```
public static Column first(Column e)
```
    Aggregate function: returns the first value in a group.
    The function by default returns the first values it sees. It will return the first non-null value it sees when ignoreNulls is set to true. If all values are null, then null is returned.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - first
```
public static Column first(java.lang.String columnName)
```
    Aggregate function: returns the first value of a column in a group.
    The function by default returns the first values it sees. It will return the first non-null value it sees when ignoreNulls is set to true. If all values are null, then null is returned.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - grouping
```
public static Column grouping(Column e)
```
    Aggregate function: indicates whether a specified column in a GROUP BY list is aggregated or not, returns 1 for aggregated or 0 for not aggregated in the result set.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    2.0.0
  - grouping
```
public static Column grouping(java.lang.String columnName)
```
    Aggregate function: indicates whether a specified column in a GROUP BY list is aggregated or not, returns 1 for aggregated or 0 for not aggregated in the result set.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    2.0.0
  - grouping_id
```
public static Column grouping_id(scala.collection.Seq<Column> cols)
```
    Aggregate function: returns the level of grouping, equals to
    (grouping(c1) << (n-1)) + (grouping(c2) << (n-2)) + ... + grouping(cn)
    Note: the list of columns should match with grouping columns exactly, or empty (means all the grouping columns).
    
    Parameters:
    cols - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    2.0.0
  - grouping_id
```
public static Column grouping_id(java.lang.String colName,
                 scala.collection.Seq<java.lang.String> colNames)
```
    Aggregate function: returns the level of grouping, equals to
    (grouping(c1) << (n-1)) + (grouping(c2) << (n-2)) + ... + grouping(cn)
    Note: the list of columns should match with grouping columns exactly.
    
    Parameters:
    colName - (undocumented)
    colNames - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    2.0.0
  - kurtosis
```
public static Column kurtosis(Column e)
```
    Aggregate function: returns the kurtosis of the values in a group.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.6.0
  - kurtosis
```
public static Column kurtosis(java.lang.String columnName)
```
    Aggregate function: returns the kurtosis of the values in a group.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.6.0
  - last
```
public static Column last(Column e,
          boolean ignoreNulls)
```
    Aggregate function: returns the last value in a group.
    The function by default returns the last values it sees. It will return the last non-null value it sees when ignoreNulls is set to true. If all values are null, then null is returned.
    
    Parameters:
    e - (undocumented)
    ignoreNulls - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    2.0.0
  - last
```
public static Column last(java.lang.String columnName,
          boolean ignoreNulls)
```
    Aggregate function: returns the last value of the column in a group.
    The function by default returns the last values it sees. It will return the last non-null value it sees when ignoreNulls is set to true. If all values are null, then null is returned.
    
    Parameters:
    columnName - (undocumented)
    ignoreNulls - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    2.0.0
  - last
```
public static Column last(Column e)
```
    Aggregate function: returns the last value in a group.
    The function by default returns the last values it sees. It will return the last non-null value it sees when ignoreNulls is set to true. If all values are null, then null is returned.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - last
```
public static Column last(java.lang.String columnName)
```
    Aggregate function: returns the last value of the column in a group.
    The function by default returns the last values it sees. It will return the last non-null value it sees when ignoreNulls is set to true. If all values are null, then null is returned.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - max
```
public static Column max(Column e)
```
    Aggregate function: returns the maximum value of the expression in a group.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - max
```
public static Column max(java.lang.String columnName)
```
    Aggregate function: returns the maximum value of the column in a group.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - mean
```
public static Column mean(Column e)
```
    Aggregate function: returns the average of the values in a group. Alias for avg.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - mean
```
public static Column mean(java.lang.String columnName)
```
    Aggregate function: returns the average of the values in a group. Alias for avg.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - min
```
public static Column min(Column e)
```
    Aggregate function: returns the minimum value of the expression in a group.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - min
```
public static Column min(java.lang.String columnName)
```
    Aggregate function: returns the minimum value of the column in a group.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - skewness
```
public static Column skewness(Column e)
```
    Aggregate function: returns the skewness of the values in a group.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.6.0
  - skewness
```
public static Column skewness(java.lang.String columnName)
```
    Aggregate function: returns the skewness of the values in a group.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.6.0
  - stddev
```
public static Column stddev(Column e)
```
    Aggregate function: alias for stddev_samp.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.6.0
  - stddev
```
public static Column stddev(java.lang.String columnName)
```
    Aggregate function: alias for stddev_samp.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.6.0
  - stddev_samp
```
public static Column stddev_samp(Column e)
```
    Aggregate function: returns the sample standard deviation of the expression in a group.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.6.0
  - stddev_samp
```
public static Column stddev_samp(java.lang.String columnName)
```
    Aggregate function: returns the sample standard deviation of the expression in a group.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.6.0
  - stddev_pop
```
public static Column stddev_pop(Column e)
```
    Aggregate function: returns the population standard deviation of the expression in a group.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.6.0
  - stddev_pop
```
public static Column stddev_pop(java.lang.String columnName)
```
    Aggregate function: returns the population standard deviation of the expression in a group.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.6.0
  - sum
```
public static Column sum(Column e)
```
    Aggregate function: returns the sum of all values in the expression.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - sum
```
public static Column sum(java.lang.String columnName)
```
    Aggregate function: returns the sum of all values in the given column.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - sumDistinct
```
public static Column sumDistinct(Column e)
```
    Aggregate function: returns the sum of distinct values in the expression.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - sumDistinct
```
public static Column sumDistinct(java.lang.String columnName)
```
    Aggregate function: returns the sum of distinct values in the expression.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - variance
```
public static Column variance(Column e)
```
    Aggregate function: alias for var_samp.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.6.0
  - variance
```
public static Column variance(java.lang.String columnName)
```
    Aggregate function: alias for var_samp.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.6.0
  - var_samp
```
public static Column var_samp(Column e)
```
    Aggregate function: returns the unbiased variance of the values in a group.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.6.0
  - var_samp
```
public static Column var_samp(java.lang.String columnName)
```
    Aggregate function: returns the unbiased variance of the values in a group.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.6.0
  - var_pop
```
public static Column var_pop(Column e)
```
    Aggregate function: returns the population variance of the values in a group.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.6.0
  - var_pop
```
public static Column var_pop(java.lang.String columnName)
```
    Aggregate function: returns the population variance of the values in a group.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.6.0
  - cume_dist
```
public static Column cume_dist()
```
    Window function: returns the cumulative distribution of values within a window partition, i.e. the fraction of rows that are below the current row.
```
   N = total number of rows in the partition
   cumeDist(x) = number of values before (and including) x / N
 
```
    Returns:
    (undocumented)
    Since:
    
    1.6.0
  - dense_rank
```
public static Column dense_rank()
```
    Window function: returns the rank of rows within a window partition, without any gaps.
    The difference between rank and denseRank is that denseRank leaves no gaps in ranking sequence when there are ties. That is, if you were ranking a competition using denseRank and had three people tie for second place, you would say that all three were in second place and that the next person came in third.
    
    Returns:
    (undocumented)
    Since:
    
    1.6.0
  - lag
```
public static Column lag(Column e,
         int offset)
```
    Window function: returns the value that is offset rows before the current row, and null if there is less than offset rows before the current row. For example, an offset of one will return the previous row at any given point in the window partition.
    This is equivalent to the LAG function in SQL.
    
    Parameters:
    e - (undocumented)
    offset - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - lag
```
public static Column lag(java.lang.String columnName,
         int offset)
```
    Window function: returns the value that is offset rows before the current row, and null if there is less than offset rows before the current row. For example, an offset of one will return the previous row at any given point in the window partition.
    This is equivalent to the LAG function in SQL.
    
    Parameters:
    columnName - (undocumented)
    offset - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - lag
```
public static Column lag(java.lang.String columnName,
         int offset,
         java.lang.Object defaultValue)
```
    Window function: returns the value that is offset rows before the current row, and defaultValue if there is less than offset rows before the current row. For example, an offset of one will return the previous row at any given point in the window partition.
    This is equivalent to the LAG function in SQL.
    
    Parameters:
    columnName - (undocumented)
    offset - (undocumented)
    defaultValue - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - lag
```
public static Column lag(Column e,
         int offset,
         java.lang.Object defaultValue)
```
    Window function: returns the value that is offset rows before the current row, and defaultValue if there is less than offset rows before the current row. For example, an offset of one will return the previous row at any given point in the window partition.
    This is equivalent to the LAG function in SQL.
    
    Parameters:
    e - (undocumented)
    offset - (undocumented)
    defaultValue - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - lead
```
public static Column lead(java.lang.String columnName,
          int offset)
```
    Window function: returns the value that is offset rows after the current row, and null if there is less than offset rows after the current row. For example, an offset of one will return the next row at any given point in the window partition.
    This is equivalent to the LEAD function in SQL.
    
    Parameters:
    columnName - (undocumented)
    offset - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - lead
```
public static Column lead(Column e,
          int offset)
```
    Window function: returns the value that is offset rows after the current row, and null if there is less than offset rows after the current row. For example, an offset of one will return the next row at any given point in the window partition.
    This is equivalent to the LEAD function in SQL.
    
    Parameters:
    e - (undocumented)
    offset - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - lead
```
public static Column lead(java.lang.String columnName,
          int offset,
          java.lang.Object defaultValue)
```
    Window function: returns the value that is offset rows after the current row, and defaultValue if there is less than offset rows after the current row. For example, an offset of one will return the next row at any given point in the window partition.
    This is equivalent to the LEAD function in SQL.
    
    Parameters:
    columnName - (undocumented)
    offset - (undocumented)
    defaultValue - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - lead
```
public static Column lead(Column e,
          int offset,
          java.lang.Object defaultValue)
```
    Window function: returns the value that is offset rows after the current row, and defaultValue if there is less than offset rows after the current row. For example, an offset of one will return the next row at any given point in the window partition.
    This is equivalent to the LEAD function in SQL.
    
    Parameters:
    e - (undocumented)
    offset - (undocumented)
    defaultValue - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - ntile
```
public static Column ntile(int n)
```
    Window function: returns the ntile group id (from 1 to n inclusive) in an ordered window partition. Fow example, if n is 4, the first quarter of the rows will get value 1, the second quarter will get 2, the third quarter will get 3, and the last quarter will get 4.
    This is equivalent to the NTILE function in SQL.
    
    Parameters:
    n - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - percent_rank
```
public static Column percent_rank()
```
    Window function: returns the relative rank (i.e. percentile) of rows within a window partition.
    This is computed by:
```
   (rank of row in its partition - 1) / (number of rows in the partition - 1)
 
```
    This is equivalent to the PERCENT_RANK function in SQL.
    Returns:
    (undocumented)
    Since:
    
    1.6.0
  - rank
```
public static Column rank()
```
    Window function: returns the rank of rows within a window partition.
    The difference between rank and denseRank is that denseRank leaves no gaps in ranking sequence when there are ties. That is, if you were ranking a competition using denseRank and had three people tie for second place, you would say that all three were in second place and that the next person came in third.
    This is equivalent to the RANK function in SQL.
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - row_number
```
public static Column row_number()
```
    Window function: returns a sequential number starting at 1 within a window partition.
    
    Returns:
    (undocumented)
    Since:
    
    1.6.0
  - abs
```
public static Column abs(Column e)
```
    Computes the absolute value.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - array
```
public static Column array(scala.collection.Seq<Column> cols)
```
    Creates a new array column. The input columns must all have the same data type.
    
    Parameters:
    cols - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - array
```
public static Column array(java.lang.String colName,
           scala.collection.Seq<java.lang.String> colNames)
```
    Creates a new array column. The input columns must all have the same data type.
    
    Parameters:
    colName - (undocumented)
    colNames - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - map
```
public static Column map(scala.collection.Seq<Column> cols)
```
    Creates a new map column. The input columns must be grouped as key-value pairs, e.g. (key1, value1, key2, value2, ...). The key columns must all have the same data type, and can't be null. The value columns must all have the same data type.
    
    Parameters:
    cols - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    2.0
  - broadcast
```
public static <T> Dataset<T> broadcast(Dataset<T> df)
```
    Marks a DataFrame as small enough for use in broadcast joins.
    The following example marks the right DataFrame for broadcast hash join using joinKey.
```
   // left and right are DataFrames
   left.join(broadcast(right), "joinKey")
 
```
    Parameters:
    df - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - coalesce
```
public static Column coalesce(scala.collection.Seq<Column> e)
```
    Returns the first column that is not null, or null if all inputs are null.
    For example, coalesce(a, b, c) will return a if a is not null, or b if a is null and b is not null, or c if both a and b are null but c is not null.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - input_file_name
```
public static Column input_file_name()
```
    Creates a string column for the file name of the current Spark task.
    
    Returns:
    (undocumented)
    Since:
    
    1.6.0
  - isnan
```
public static Column isnan(Column e)
```
    Return true iff the column is NaN.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.6.0
  - isnull
```
public static Column isnull(Column e)
```
    Return true iff the column is null.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.6.0
  - monotonicallyIncreasingId
```
public static Column monotonicallyIncreasingId()
```
    A column expression that generates monotonically increasing 64-bit integers.
    The generated ID is guaranteed to be monotonically increasing and unique, but not consecutive. The current implementation puts the partition ID in the upper 31 bits, and the record number within each partition in the lower 33 bits. The assumption is that the data frame has less than 1 billion partitions, and each partition has less than 8 billion records.
    As an example, consider a DataFrame with two partitions, each with 3 records. This expression would return the following IDs: 0, 1, 2, 8589934592 (1L << 33), 8589934593, 8589934594.
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - monotonically_increasing_id
```
public static Column monotonically_increasing_id()
```
    A column expression that generates monotonically increasing 64-bit integers.
    The generated ID is guaranteed to be monotonically increasing and unique, but not consecutive. The current implementation puts the partition ID in the upper 31 bits, and the record number within each partition in the lower 33 bits. The assumption is that the data frame has less than 1 billion partitions, and each partition has less than 8 billion records.
    As an example, consider a DataFrame with two partitions, each with 3 records. This expression would return the following IDs: 0, 1, 2, 8589934592 (1L << 33), 8589934593, 8589934594.
    
    Returns:
    (undocumented)
    Since:
    
    1.6.0
  - nanvl
```
public static Column nanvl(Column col1,
           Column col2)
```
    Returns col1 if it is not NaN, or col2 if col1 is NaN.
    Both inputs should be floating point columns (DoubleType or FloatType).
    
    Parameters:
    col1 - (undocumented)
    col2 - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - negate
```
public static Column negate(Column e)
```
    Unary minus, i.e. negate the expression.
```
   // Select the amount column and negates all values.
   // Scala:
   df.select( -df("amount") )

   // Java:
   df.select( negate(df.col("amount")) );
 
```
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - not
```
public static Column not(Column e)
```
    Inversion of boolean expression, i.e. NOT.
```
   // Scala: select rows that are not active (isActive === false)
   df.filter( !df("isActive") )

   // Java:
   df.filter( not(df.col("isActive")) );
 
```
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - rand
```
public static Column rand(long seed)
```
    Generate a random column with i.i.d. samples from U[0.0, 1.0].
    Note that this is indeterministic when data partitions are not fixed.
    
    Parameters:
    seed - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - rand
```
public static Column rand()
```
    Generate a random column with i.i.d. samples from U[0.0, 1.0].
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - randn
```
public static Column randn(long seed)
```
    Generate a column with i.i.d. samples from the standard normal distribution.
    Note that this is indeterministic when data partitions are not fixed.
    
    Parameters:
    seed - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - randn
```
public static Column randn()
```
    Generate a column with i.i.d. samples from the standard normal distribution.
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - spark_partition_id
```
public static Column spark_partition_id()
```
    Partition ID of the Spark task.
    Note that this is indeterministic because it depends on data partitioning and task scheduling.
    
    Returns:
    (undocumented)
    Since:
    
    1.6.0
  - sqrt
```
public static Column sqrt(Column e)
```
    Computes the square root of the specified float value.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - sqrt
```
public static Column sqrt(java.lang.String colName)
```
    Computes the square root of the specified float value.
    
    Parameters:
    colName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - struct
```
public static Column struct(scala.collection.Seq<Column> cols)
```
    Creates a new struct column. If the input column is a column in a DataFrame, or a derived column expression that is named (i.e. aliased), its name would be remained as the StructField's name, otherwise, the newly generated StructField's name would be auto generated as col${index + 1}, i.e. col1, col2, col3, ...
    
    Parameters:
    cols - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - struct
```
public static Column struct(java.lang.String colName,
            scala.collection.Seq<java.lang.String> colNames)
```
    Creates a new struct column that composes multiple input columns.
    
    Parameters:
    colName - (undocumented)
    colNames - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - when
```
public static Column when(Column condition,
          java.lang.Object value)
```
    Evaluates a list of conditions and returns one of multiple possible result expressions. If otherwise is not defined at the end, null is returned for unmatched conditions.
```
   // Example: encoding gender string column into integer.

   // Scala:
   people.select(when(people("gender") === "male", 0)
     .when(people("gender") === "female", 1)
     .otherwise(2))

   // Java:
   people.select(when(col("gender").equalTo("male"), 0)
     .when(col("gender").equalTo("female"), 1)
     .otherwise(2))
 
```
    Parameters:
    condition - (undocumented)
    value - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - bitwiseNOT
```
public static Column bitwiseNOT(Column e)
```
    Computes bitwise NOT.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - expr
```
public static Column expr(java.lang.String expr)
```
    Parses the expression string into the column that it represents, similar to DataFrame.selectExpr
```
   // get the number of words of each length
   df.groupBy(expr("length(word)")).count()
 
```
    Parameters:
    expr - (undocumented)
    
    Returns:
    (undocumented)
  - acos
```
public static Column acos(Column e)
```
    Computes the cosine inverse of the given value; the returned angle is in the range 0.0 through pi.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - acos
```
public static Column acos(java.lang.String columnName)
```
    Computes the cosine inverse of the given column; the returned angle is in the range 0.0 through pi.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - asin
```
public static Column asin(Column e)
```
    Computes the sine inverse of the given value; the returned angle is in the range -pi/2 through pi/2.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - asin
```
public static Column asin(java.lang.String columnName)
```
    Computes the sine inverse of the given column; the returned angle is in the range -pi/2 through pi/2.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - atan
```
public static Column atan(Column e)
```
    Computes the tangent inverse of the given value.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - atan
```
public static Column atan(java.lang.String columnName)
```
    Computes the tangent inverse of the given column.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - atan2
```
public static Column atan2(Column l,
           Column r)
```
    Returns the angle theta from the conversion of rectangular coordinates (x, y) to polar coordinates (r, theta).
    
    Parameters:
    l - (undocumented)
    r - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - atan2
```
public static Column atan2(Column l,
           java.lang.String rightName)
```
    Returns the angle theta from the conversion of rectangular coordinates (x, y) to polar coordinates (r, theta).
    
    Parameters:
    l - (undocumented)
    rightName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - atan2
```
public static Column atan2(java.lang.String leftName,
           Column r)
```
    Returns the angle theta from the conversion of rectangular coordinates (x, y) to polar coordinates (r, theta).
    
    Parameters:
    leftName - (undocumented)
    r - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - atan2
```
public static Column atan2(java.lang.String leftName,
           java.lang.String rightName)
```
    Returns the angle theta from the conversion of rectangular coordinates (x, y) to polar coordinates (r, theta).
    
    Parameters:
    leftName - (undocumented)
    rightName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - atan2
```
public static Column atan2(Column l,
           double r)
```
    Returns the angle theta from the conversion of rectangular coordinates (x, y) to polar coordinates (r, theta).
    
    Parameters:
    l - (undocumented)
    r - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - atan2
```
public static Column atan2(java.lang.String leftName,
           double r)
```
    Returns the angle theta from the conversion of rectangular coordinates (x, y) to polar coordinates (r, theta).
    
    Parameters:
    leftName - (undocumented)
    r - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - atan2
```
public static Column atan2(double l,
           Column r)
```
    Returns the angle theta from the conversion of rectangular coordinates (x, y) to polar coordinates (r, theta).
    
    Parameters:
    l - (undocumented)
    r - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - atan2
```
public static Column atan2(double l,
           java.lang.String rightName)
```
    Returns the angle theta from the conversion of rectangular coordinates (x, y) to polar coordinates (r, theta).
    
    Parameters:
    l - (undocumented)
    rightName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - bin
```
public static Column bin(Column e)
```
    An expression that returns the string representation of the binary value of the given long column. For example, bin("12") returns "1100".
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - bin
```
public static Column bin(java.lang.String columnName)
```
    An expression that returns the string representation of the binary value of the given long column. For example, bin("12") returns "1100".
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - cbrt
```
public static Column cbrt(Column e)
```
    Computes the cube-root of the given value.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - cbrt
```
public static Column cbrt(java.lang.String columnName)
```
    Computes the cube-root of the given column.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - ceil
```
public static Column ceil(Column e)
```
    Computes the ceiling of the given value.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - ceil
```
public static Column ceil(java.lang.String columnName)
```
    Computes the ceiling of the given column.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - conv
```
public static Column conv(Column num,
          int fromBase,
          int toBase)
```
    Convert a number in a string column from one base to another.
    
    Parameters:
    num - (undocumented)
    fromBase - (undocumented)
    toBase - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - cos
```
public static Column cos(Column e)
```
    Computes the cosine of the given value.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - cos
```
public static Column cos(java.lang.String columnName)
```
    Computes the cosine of the given column.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - cosh
```
public static Column cosh(Column e)
```
    Computes the hyperbolic cosine of the given value.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - cosh
```
public static Column cosh(java.lang.String columnName)
```
    Computes the hyperbolic cosine of the given column.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - exp
```
public static Column exp(Column e)
```
    Computes the exponential of the given value.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - exp
```
public static Column exp(java.lang.String columnName)
```
    Computes the exponential of the given column.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - expm1
```
public static Column expm1(Column e)
```
    Computes the exponential of the given value minus one.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - expm1
```
public static Column expm1(java.lang.String columnName)
```
    Computes the exponential of the given column.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - factorial
```
public static Column factorial(Column e)
```
    Computes the factorial of the given value.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - floor
```
public static Column floor(Column e)
```
    Computes the floor of the given value.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - floor
```
public static Column floor(java.lang.String columnName)
```
    Computes the floor of the given column.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - greatest
```
public static Column greatest(scala.collection.Seq<Column> exprs)
```
    Returns the greatest value of the list of values, skipping null values. This function takes at least 2 parameters. It will return null iff all parameters are null.
    
    Parameters:
    exprs - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - greatest
```
public static Column greatest(java.lang.String columnName,
              scala.collection.Seq<java.lang.String> columnNames)
```
    Returns the greatest value of the list of column names, skipping null values. This function takes at least 2 parameters. It will return null iff all parameters are null.
    
    Parameters:
    columnName - (undocumented)
    columnNames - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - hex
```
public static Column hex(Column column)
```
    Computes hex value of the given column.
    
    Parameters:
    column - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - unhex
```
public static Column unhex(Column column)
```
    Inverse of hex. Interprets each pair of characters as a hexadecimal number and converts to the byte representation of number.
    
    Parameters:
    column - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - hypot
```
public static Column hypot(Column l,
           Column r)
```
    Computes sqrt(a^2^ + b^2^) without intermediate overflow or underflow.
    
    Parameters:
    l - (undocumented)
    r - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - hypot
```
public static Column hypot(Column l,
           java.lang.String rightName)
```
    Computes sqrt(a^2^ + b^2^) without intermediate overflow or underflow.
    
    Parameters:
    l - (undocumented)
    rightName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - hypot
```
public static Column hypot(java.lang.String leftName,
           Column r)
```
    Computes sqrt(a^2^ + b^2^) without intermediate overflow or underflow.
    
    Parameters:
    leftName - (undocumented)
    r - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - hypot
```
public static Column hypot(java.lang.String leftName,
           java.lang.String rightName)
```
    Computes sqrt(a^2^ + b^2^) without intermediate overflow or underflow.
    
    Parameters:
    leftName - (undocumented)
    rightName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - hypot
```
public static Column hypot(Column l,
           double r)
```
    Computes sqrt(a^2^ + b^2^) without intermediate overflow or underflow.
    
    Parameters:
    l - (undocumented)
    r - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - hypot
```
public static Column hypot(java.lang.String leftName,
           double r)
```
    Computes sqrt(a^2^ + b^2^) without intermediate overflow or underflow.
    
    Parameters:
    leftName - (undocumented)
    r - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - hypot
```
public static Column hypot(double l,
           Column r)
```
    Computes sqrt(a^2^ + b^2^) without intermediate overflow or underflow.
    
    Parameters:
    l - (undocumented)
    r - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - hypot
```
public static Column hypot(double l,
           java.lang.String rightName)
```
    Computes sqrt(a^2^ + b^2^) without intermediate overflow or underflow.
    
    Parameters:
    l - (undocumented)
    rightName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - least
```
public static Column least(scala.collection.Seq<Column> exprs)
```
    Returns the least value of the list of values, skipping null values. This function takes at least 2 parameters. It will return null iff all parameters are null.
    
    Parameters:
    exprs - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - least
```
public static Column least(java.lang.String columnName,
           scala.collection.Seq<java.lang.String> columnNames)
```
    Returns the least value of the list of column names, skipping null values. This function takes at least 2 parameters. It will return null iff all parameters are null.
    
    Parameters:
    columnName - (undocumented)
    columnNames - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - log
```
public static Column log(Column e)
```
    Computes the natural logarithm of the given value.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - log
```
public static Column log(java.lang.String columnName)
```
    Computes the natural logarithm of the given column.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - log
```
public static Column log(double base,
         Column a)
```
    Returns the first argument-base logarithm of the second argument.
    
    Parameters:
    base - (undocumented)
    a - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - log
```
public static Column log(double base,
         java.lang.String columnName)
```
    Returns the first argument-base logarithm of the second argument.
    
    Parameters:
    base - (undocumented)
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - log10
```
public static Column log10(Column e)
```
    Computes the logarithm of the given value in base 10.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - log10
```
public static Column log10(java.lang.String columnName)
```
    Computes the logarithm of the given value in base 10.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - log1p
```
public static Column log1p(Column e)
```
    Computes the natural logarithm of the given value plus one.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - log1p
```
public static Column log1p(java.lang.String columnName)
```
    Computes the natural logarithm of the given column plus one.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - log2
```
public static Column log2(Column expr)
```
    Computes the logarithm of the given column in base 2.
    
    Parameters:
    expr - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - log2
```
public static Column log2(java.lang.String columnName)
```
    Computes the logarithm of the given value in base 2.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - pow
```
public static Column pow(Column l,
         Column r)
```
    Returns the value of the first argument raised to the power of the second argument.
    
    Parameters:
    l - (undocumented)
    r - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - pow
```
public static Column pow(Column l,
         java.lang.String rightName)
```
    Returns the value of the first argument raised to the power of the second argument.
    
    Parameters:
    l - (undocumented)
    rightName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - pow
```
public static Column pow(java.lang.String leftName,
         Column r)
```
    Returns the value of the first argument raised to the power of the second argument.
    
    Parameters:
    leftName - (undocumented)
    r - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - pow
```
public static Column pow(java.lang.String leftName,
         java.lang.String rightName)
```
    Returns the value of the first argument raised to the power of the second argument.
    
    Parameters:
    leftName - (undocumented)
    rightName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - pow
```
public static Column pow(Column l,
         double r)
```
    Returns the value of the first argument raised to the power of the second argument.
    
    Parameters:
    l - (undocumented)
    r - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - pow
```
public static Column pow(java.lang.String leftName,
         double r)
```
    Returns the value of the first argument raised to the power of the second argument.
    
    Parameters:
    leftName - (undocumented)
    r - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - pow
```
public static Column pow(double l,
         Column r)
```
    Returns the value of the first argument raised to the power of the second argument.
    
    Parameters:
    l - (undocumented)
    r - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - pow
```
public static Column pow(double l,
         java.lang.String rightName)
```
    Returns the value of the first argument raised to the power of the second argument.
    
    Parameters:
    l - (undocumented)
    rightName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - pmod
```
public static Column pmod(Column dividend,
          Column divisor)
```
    Returns the positive value of dividend mod divisor.
    
    Parameters:
    dividend - (undocumented)
    divisor - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - rint
```
public static Column rint(Column e)
```
    Returns the double value that is closest in value to the argument and is equal to a mathematical integer.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - rint
```
public static Column rint(java.lang.String columnName)
```
    Returns the double value that is closest in value to the argument and is equal to a mathematical integer.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - round
```
public static Column round(Column e)
```
    Returns the value of the column e rounded to 0 decimal places.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - round
```
public static Column round(Column e,
           int scale)
```
    Round the value of e to scale decimal places if scale >= 0 or at integral part when scale < 0.
    
    Parameters:
    e - (undocumented)
    scale - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - bround
```
public static Column bround(Column e)
```
    Returns the value of the column e rounded to 0 decimal places with HALF_EVEN round mode.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    2.0.0
  - bround
```
public static Column bround(Column e,
            int scale)
```
    Round the value of e to scale decimal places with HALF_EVEN round mode if scale >= 0 or at integral part when scale < 0.
    
    Parameters:
    e - (undocumented)
    scale - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    2.0.0
  - shiftLeft
```
public static Column shiftLeft(Column e,
               int numBits)
```
    Shift the given value numBits left. If the given value is a long value, this function will return a long value else it will return an integer value.
    
    Parameters:
    e - (undocumented)
    numBits - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - shiftRight
```
public static Column shiftRight(Column e,
                int numBits)
```
    Shift the given value numBits right. If the given value is a long value, it will return a long value else it will return an integer value.
    
    Parameters:
    e - (undocumented)
    numBits - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - shiftRightUnsigned
```
public static Column shiftRightUnsigned(Column e,
                        int numBits)
```
    Unsigned shift the given value numBits right. If the given value is a long value, it will return a long value else it will return an integer value.
    
    Parameters:
    e - (undocumented)
    numBits - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - signum
```
public static Column signum(Column e)
```
    Computes the signum of the given value.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - signum
```
public static Column signum(java.lang.String columnName)
```
    Computes the signum of the given column.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - sin
```
public static Column sin(Column e)
```
    Computes the sine of the given value.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - sin
```
public static Column sin(java.lang.String columnName)
```
    Computes the sine of the given column.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - sinh
```
public static Column sinh(Column e)
```
    Computes the hyperbolic sine of the given value.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - sinh
```
public static Column sinh(java.lang.String columnName)
```
    Computes the hyperbolic sine of the given column.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - tan
```
public static Column tan(Column e)
```
    Computes the tangent of the given value.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - tan
```
public static Column tan(java.lang.String columnName)
```
    Computes the tangent of the given column.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - tanh
```
public static Column tanh(Column e)
```
    Computes the hyperbolic tangent of the given value.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - tanh
```
public static Column tanh(java.lang.String columnName)
```
    Computes the hyperbolic tangent of the given column.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - toDegrees
```
public static Column toDegrees(Column e)
```
    Converts an angle measured in radians to an approximately equivalent angle measured in degrees.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - toDegrees
```
public static Column toDegrees(java.lang.String columnName)
```
    Converts an angle measured in radians to an approximately equivalent angle measured in degrees.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - toRadians
```
public static Column toRadians(Column e)
```
    Converts an angle measured in degrees to an approximately equivalent angle measured in radians.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - toRadians
```
public static Column toRadians(java.lang.String columnName)
```
    Converts an angle measured in degrees to an approximately equivalent angle measured in radians.
    
    Parameters:
    columnName - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.4.0
  - md5
```
public static Column md5(Column e)
```
    Calculates the MD5 digest of a binary column and returns the value as a 32 character hex string.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - sha1
```
public static Column sha1(Column e)
```
    Calculates the SHA-1 digest of a binary column and returns the value as a 40 character hex string.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - sha2
```
public static Column sha2(Column e,
          int numBits)
```
    Calculates the SHA-2 family of hash functions of a binary column and returns the value as a hex string.
    
    Parameters:
    e - column to compute SHA-2 on.
    numBits - one of 224, 256, 384, or 512.
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - crc32
```
public static Column crc32(Column e)
```
    Calculates the cyclic redundancy check value (CRC32) of a binary column and returns the value as a bigint.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - hash
```
public static Column hash(scala.collection.Seq<Column> cols)
```
    Calculates the hash code of given columns, and returns the result as an int column.
    
    Parameters:
    cols - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    2.0
  - ascii
```
public static Column ascii(Column e)
```
    Computes the numeric value of the first character of the string column, and returns the result as a int column.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - base64
```
public static Column base64(Column e)
```
    Computes the BASE64 encoding of a binary column and returns it as a string column. This is the reverse of unbase64.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - concat
```
public static Column concat(scala.collection.Seq<Column> exprs)
```
    Concatenates multiple input string columns together into a single string column.
    
    Parameters:
    exprs - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - concat_ws
```
public static Column concat_ws(java.lang.String sep,
               scala.collection.Seq<Column> exprs)
```
    Concatenates multiple input string columns together into a single string column, using the given separator.
    
    Parameters:
    sep - (undocumented)
    exprs - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - decode
```
public static Column decode(Column value,
            java.lang.String charset)
```
    Computes the first argument into a string from a binary using the provided character set (one of 'US-ASCII', 'ISO-8859-1', 'UTF-8', 'UTF-16BE', 'UTF-16LE', 'UTF-16'). If either argument is null, the result will also be null.
    
    Parameters:
    value - (undocumented)
    charset - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - encode
```
public static Column encode(Column value,
            java.lang.String charset)
```
    Computes the first argument into a binary from a string using the provided character set (one of 'US-ASCII', 'ISO-8859-1', 'UTF-8', 'UTF-16BE', 'UTF-16LE', 'UTF-16'). If either argument is null, the result will also be null.
    
    Parameters:
    value - (undocumented)
    charset - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - format_number
```
public static Column format_number(Column x,
                   int d)
```
    Formats numeric column x to a format like '#,###,###.##', rounded to d decimal places, and returns the result as a string column.
    If d is 0, the result has no decimal point or fractional part. If d < 0, the result will be null.
    
    Parameters:
    x - (undocumented)
    d - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - format_string
```
public static Column format_string(java.lang.String format,
                   scala.collection.Seq<Column> arguments)
```
    Formats the arguments in printf-style and returns the result as a string column.
    
    Parameters:
    format - (undocumented)
    arguments - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - initcap
```
public static Column initcap(Column e)
```
    Returns a new string column by converting the first letter of each word to uppercase. Words are delimited by whitespace.
    For example, "hello world" will become "Hello World".
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - instr
```
public static Column instr(Column str,
           java.lang.String substring)
```
    Locate the position of the first occurrence of substr column in the given string. Returns null if either of the arguments are null.
    NOTE: The position is not zero based, but 1 based index, returns 0 if substr could not be found in str.
    
    Parameters:
    str - (undocumented)
    substring - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - length
```
public static Column length(Column e)
```
    Computes the length of a given string or binary column.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - lower
```
public static Column lower(Column e)
```
    Converts a string column to lower case.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - levenshtein
```
public static Column levenshtein(Column l,
                 Column r)
```
    Computes the Levenshtein distance of the two given string columns.
    
    Parameters:
    l - (undocumented)
    r - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - locate
```
public static Column locate(java.lang.String substr,
            Column str)
```
    Locate the position of the first occurrence of substr. NOTE: The position is not zero based, but 1 based index, returns 0 if substr could not be found in str.
    
    Parameters:
    substr - (undocumented)
    str - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - locate
```
public static Column locate(java.lang.String substr,
            Column str,
            int pos)
```
    Locate the position of the first occurrence of substr in a string column, after position pos.
    NOTE: The position is not zero based, but 1 based index. returns 0 if substr could not be found in str.
    
    Parameters:
    substr - (undocumented)
    str - (undocumented)
    pos - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - lpad
```
public static Column lpad(Column str,
          int len,
          java.lang.String pad)
```
    Left-pad the string column with
    
    Parameters:
    str - (undocumented)
    len - (undocumented)
    pad - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - ltrim
```
public static Column ltrim(Column e)
```
    Trim the spaces from left end for the specified string value.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - regexp_extract
```
public static Column regexp_extract(Column e,
                    java.lang.String exp,
                    int groupIdx)
```
    Extract a specific(idx) group identified by a java regex, from the specified string column.
    
    Parameters:
    e - (undocumented)
    exp - (undocumented)
    groupIdx - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - regexp_replace
```
public static Column regexp_replace(Column e,
                    java.lang.String pattern,
                    java.lang.String replacement)
```
    Replace all substrings of the specified string value that match regexp with rep.
    
    Parameters:
    e - (undocumented)
    pattern - (undocumented)
    replacement - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - unbase64
```
public static Column unbase64(Column e)
```
    Decodes a BASE64 encoded string column and returns it as a binary column. This is the reverse of base64.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - rpad
```
public static Column rpad(Column str,
          int len,
          java.lang.String pad)
```
    Right-padded with pad to a length of len.
    
    Parameters:
    str - (undocumented)
    len - (undocumented)
    pad - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - repeat
```
public static Column repeat(Column str,
            int n)
```
    Repeats a string column n times, and returns it as a new string column.
    
    Parameters:
    str - (undocumented)
    n - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - reverse
```
public static Column reverse(Column str)
```
    Reverses the string column and returns it as a new string column.
    
    Parameters:
    str - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - rtrim
```
public static Column rtrim(Column e)
```
    Trim the spaces from right end for the specified string value.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - soundex
```
public static Column soundex(Column e)
```
    * Return the soundex code for the specified expression.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - split
```
public static Column split(Column str,
           java.lang.String pattern)
```
    Splits str around pattern (pattern is a regular expression). NOTE: pattern is a string representation of the regular expression.
    
    Parameters:
    str - (undocumented)
    pattern - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - substring
```
public static Column substring(Column str,
               int pos,
               int len)
```
    Substring starts at pos and is of length len when str is String type or returns the slice of byte array that starts at pos in byte and is of length len when str is Binary type
    
    Parameters:
    str - (undocumented)
    pos - (undocumented)
    len - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - substring_index
```
public static Column substring_index(Column str,
                     java.lang.String delim,
                     int count)
```
    Returns the substring from string str before count occurrences of the delimiter delim. If count is positive, everything the left of the final delimiter (counting from left) is returned. If count is negative, every to the right of the final delimiter (counting from the right) is returned. substring_index performs a case-sensitive match when searching for delim.
    
    Parameters:
    str - (undocumented)
    delim - (undocumented)
    count - (undocumented)
    
    Returns:
    (undocumented)
  - translate
```
public static Column translate(Column src,
               java.lang.String matchingString,
               java.lang.String replaceString)
```
    Translate any character in the src by a character in replaceString. The characters in replaceString correspond to the characters in matchingString. The translate will happen when any character in the string matches the character in the matchingString.
    
    Parameters:
    src - (undocumented)
    matchingString - (undocumented)
    replaceString - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - trim
```
public static Column trim(Column e)
```
    Trim the spaces from both ends for the specified string column.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - upper
```
public static Column upper(Column e)
```
    Converts a string column to upper case.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - add_months
```
public static Column add_months(Column startDate,
                int numMonths)
```
    Returns the date that is numMonths after startDate.
    
    Parameters:
    startDate - (undocumented)
    numMonths - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - current_date
```
public static Column current_date()
```
    Returns the current date as a date column.
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - current_timestamp
```
public static Column current_timestamp()
```
    Returns the current timestamp as a timestamp column.
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - date_format
```
public static Column date_format(Column dateExpr,
                 java.lang.String format)
```
    Converts a date/timestamp/string to a value of string in the format specified by the date format given by the second argument.
    A pattern could be for instance dd.MM.yyyy and could return a string like '18.03.1993'. All pattern letters of SimpleDateFormat can be used.
    NOTE: Use when ever possible specialized functions like year. These benefit from a specialized implementation.
    
    Parameters:
    dateExpr - (undocumented)
    format - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - date_add
```
public static Column date_add(Column start,
              int days)
```
    Returns the date that is days days after start
    
    Parameters:
    start - (undocumented)
    days - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - date_sub
```
public static Column date_sub(Column start,
              int days)
```
    Returns the date that is days days before start
    
    Parameters:
    start - (undocumented)
    days - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - datediff
```
public static Column datediff(Column end,
              Column start)
```
    Returns the number of days from start to end.
    
    Parameters:
    end - (undocumented)
    start - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - year
```
public static Column year(Column e)
```
    Extracts the year as an integer from a given date/timestamp/string.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - quarter
```
public static Column quarter(Column e)
```
    Extracts the quarter as an integer from a given date/timestamp/string.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - month
```
public static Column month(Column e)
```
    Extracts the month as an integer from a given date/timestamp/string.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - dayofmonth
```
public static Column dayofmonth(Column e)
```
    Extracts the day of the month as an integer from a given date/timestamp/string.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - dayofyear
```
public static Column dayofyear(Column e)
```
    Extracts the day of the year as an integer from a given date/timestamp/string.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - hour
```
public static Column hour(Column e)
```
    Extracts the hours as an integer from a given date/timestamp/string.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - last_day
```
public static Column last_day(Column e)
```
    Given a date column, returns the last day of the month which the given date belongs to. For example, input "2015-07-27" returns "2015-07-31" since July 31 is the last day of the month in July 2015.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - minute
```
public static Column minute(Column e)
```
    Extracts the minutes as an integer from a given date/timestamp/string.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - months_between
```
public static Column months_between(Column date1,
                    Column date2)
```
  - next_day
```
public static Column next_day(Column date,
              java.lang.String dayOfWeek)
```
    Given a date column, returns the first date which is later than the value of the date column that is on the specified day of the week.
    For example, next_day('2015-07-27', "Sunday") returns 2015-08-02 because that is the first Sunday after 2015-07-27.
    Day of the week parameter is case insensitive, and accepts: "Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun".
    
    Parameters:
    date - (undocumented)
    dayOfWeek - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - second
```
public static Column second(Column e)
```
    Extracts the seconds as an integer from a given date/timestamp/string.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - weekofyear
```
public static Column weekofyear(Column e)
```
    Extracts the week number as an integer from a given date/timestamp/string.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - from_unixtime
```
public static Column from_unixtime(Column ut)
```
    Converts the number of seconds from unix epoch (1970-01-01 00:00:00 UTC) to a string representing the timestamp of that moment in the current system time zone in the given format.
    
    Parameters:
    ut - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - from_unixtime
```
public static Column from_unixtime(Column ut,
                   java.lang.String f)
```
    Converts the number of seconds from unix epoch (1970-01-01 00:00:00 UTC) to a string representing the timestamp of that moment in the current system time zone in the given format.
    
    Parameters:
    ut - (undocumented)
    f - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - unix_timestamp
```
public static Column unix_timestamp()
```
    Gets current Unix timestamp in seconds.
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - unix_timestamp
```
public static Column unix_timestamp(Column s)
```
    Converts time string in format yyyy-MM-dd HH:mm:ss to Unix timestamp (in seconds), using the default timezone and the default locale, return null if fail.
    
    Parameters:
    s - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - unix_timestamp
```
public static Column unix_timestamp(Column s,
                    java.lang.String p)
```
    Convert time string with given pattern (see [http://docs.oracle.com/javase/tutorial/i18n/format/simpleDateFormat.html]) to Unix time stamp (in seconds), return null if fail.
    
    Parameters:
    s - (undocumented)
    p - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - to_date
```
public static Column to_date(Column e)
```
    Converts the column into DateType.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - trunc
```
public static Column trunc(Column date,
           java.lang.String format)
```
    Returns date truncated to the unit specified by the format.
    
    Parameters:
    format: - 'year', 'yyyy', 'yy' for truncate by year, or 'month', 'mon', 'mm' for truncate by month
    date - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - from_utc_timestamp
```
public static Column from_utc_timestamp(Column ts,
                        java.lang.String tz)
```
    Assumes given timestamp is UTC and converts to given timezone.
    
    Parameters:
    ts - (undocumented)
    tz - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - to_utc_timestamp
```
public static Column to_utc_timestamp(Column ts,
                      java.lang.String tz)
```
    Assumes given timestamp is in given timezone and converts to UTC.
    
    Parameters:
    ts - (undocumented)
    tz - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - window
```
public static Column window(Column timeColumn,
            java.lang.String windowDuration,
            java.lang.String slideDuration,
            java.lang.String startTime)
```
    Bucketize rows into one or more time windows given a timestamp specifying column. Window starts are inclusive but the window ends are exclusive, e.g. 12:05 will be in the window [12:05,12:10) but not in [12:00,12:05). Windows can support microsecond precision. Windows in the order of months are not supported. The following example takes the average stock price for a one minute window every 10 seconds starting 5 seconds after the hour:
```
   val df = ... // schema => timestamp: TimestampType, stockId: StringType, price: DoubleType
   df.groupBy(window($"time", "1 minute", "10 seconds", "5 seconds"), $"stockId")
     .agg(mean("price"))
 
```
    The windows will look like:
```
   09:00:05-09:01:05
   09:00:15-09:01:15
   09:00:25-09:01:25 ...
 
```
    For a continuous query, you may use the function current_timestamp to generate windows on processing time.
    Parameters:
    timeColumn - The column or the expression to use as the timestamp for windowing by time. The time column must be of TimestampType.
    windowDuration - A string specifying the width of the window, e.g. 10 minutes, 1 second. Check CalendarInterval for valid duration identifiers.
    slideDuration - A string specifying the sliding interval of the window, e.g. 1 minute. A new window will be generated every slideDuration. Must be less than or equal to the windowDuration. Check CalendarInterval for valid duration identifiers.
    startTime - The offset with respect to 1970-01-01 00:00:00 UTC with which to start window intervals. For example, in order to have hourly tumbling windows that start 15 minutes past the hour, e.g. 12:15-13:15, 13:15-14:15... provide startTime as 15 minutes.
    
    Returns:
    (undocumented)
    Since:
    
    2.0.0
  - window
```
public static Column window(Column timeColumn,
            java.lang.String windowDuration,
            java.lang.String slideDuration)
```
    Bucketize rows into one or more time windows given a timestamp specifying column. Window starts are inclusive but the window ends are exclusive, e.g. 12:05 will be in the window [12:05,12:10) but not in [12:00,12:05). Windows can support microsecond precision. Windows in the order of months are not supported. The windows start beginning at 1970-01-01 00:00:00 UTC. The following example takes the average stock price for a one minute window every 10 seconds:
```
   val df = ... // schema => timestamp: TimestampType, stockId: StringType, price: DoubleType
   df.groupBy(window($"time", "1 minute", "10 seconds"), $"stockId")
     .agg(mean("price"))
 
```
    The windows will look like:
```
   09:00:00-09:01:00
   09:00:10-09:01:10
   09:00:20-09:01:20 ...
 
```
    For a continuous query, you may use the function current_timestamp to generate windows on processing time.
    Parameters:
    timeColumn - The column or the expression to use as the timestamp for windowing by time. The time column must be of TimestampType.
    windowDuration - A string specifying the width of the window, e.g. 10 minutes, 1 second. Check CalendarInterval for valid duration identifiers.
    slideDuration - A string specifying the sliding interval of the window, e.g. 1 minute. A new window will be generated every slideDuration. Must be less than or equal to the windowDuration. Check CalendarInterval for valid duration.
    
    Returns:
    (undocumented)
    Since:
    
    2.0.0
  - window
```
public static Column window(Column timeColumn,
            java.lang.String windowDuration)
```
    Generates tumbling time windows given a timestamp specifying column. Window starts are inclusive but the window ends are exclusive, e.g. 12:05 will be in the window [12:05,12:10) but not in [12:00,12:05). Windows can support microsecond precision. Windows in the order of months are not supported. The windows start beginning at 1970-01-01 00:00:00 UTC. The following example takes the average stock price for a one minute tumbling window:
```
   val df = ... // schema => timestamp: TimestampType, stockId: StringType, price: DoubleType
   df.groupBy(window($"time", "1 minute"), $"stockId")
     .agg(mean("price"))
 
```
    The windows will look like:
```
   09:00:00-09:01:00
   09:01:00-09:02:00
   09:02:00-09:03:00 ...
 
```
    For a continuous query, you may use the function current_timestamp to generate windows on processing time.
    Parameters:
    timeColumn - The column or the expression to use as the timestamp for windowing by time. The time column must be of TimestampType.
    windowDuration - A string specifying the width of the window, e.g. 10 minutes, 1 second. Check CalendarInterval for valid duration identifiers.
    
    Returns:
    (undocumented)
    Since:
    
    2.0.0
  - array_contains
```
public static Column array_contains(Column column,
                    java.lang.Object value)
```
    Returns true if the array contains value
    
    Parameters:
    column - (undocumented)
    value - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - explode
```
public static Column explode(Column e)
```
    Creates a new row for each element in the given array or map column.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - get_json_object
```
public static Column get_json_object(Column e,
                     java.lang.String path)
```
    Extracts json object from a json string based on json path specified, and returns json string of the extracted json object. It will return null if the input json string is invalid.
    
    Parameters:
    e - (undocumented)
    path - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.6.0
  - json_tuple
```
public static Column json_tuple(Column json,
                scala.collection.Seq<java.lang.String> fields)
```
    Creates a new row for a json column according to the given field names.
    
    Parameters:
    json - (undocumented)
    fields - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.6.0
  - size
```
public static Column size(Column e)
```
    Returns length of array or map.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - sort_array
```
public static Column sort_array(Column e)
```
    Sorts the input array for the given column in ascending order, according to the natural ordering of the array elements.
    
    Parameters:
    e - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - sort_array
```
public static Column sort_array(Column e,
                boolean asc)
```
    Sorts the input array for the given column in ascending / descending order, according to the natural ordering of the array elements.
    
    Parameters:
    e - (undocumented)
    asc - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0
  - udf
```
public static <RT> UserDefinedFunction udf(scala.Function0<RT> f,
                           scala.reflect.api.TypeTags.TypeTag<RT> evidence$1)
```
    Defines a user-defined function of 0 arguments as user-defined function (UDF). The data types are automatically inferred based on the function's signature.
    
    Parameters:
    f - (undocumented)
    evidence$1 - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - udf
```
public static <RT,A1> UserDefinedFunction udf(scala.Function1<A1,RT> f,
                              scala.reflect.api.TypeTags.TypeTag<RT> evidence$2,
                              scala.reflect.api.TypeTags.TypeTag<A1> evidence$3)
```
    Defines a user-defined function of 1 arguments as user-defined function (UDF). The data types are automatically inferred based on the function's signature.
    
    Parameters:
    f - (undocumented)
    evidence$2 - (undocumented)
    evidence$3 - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - udf
```
public static <RT,A1,A2> UserDefinedFunction udf(scala.Function2<A1,A2,RT> f,
                                 scala.reflect.api.TypeTags.TypeTag<RT> evidence$4,
                                 scala.reflect.api.TypeTags.TypeTag<A1> evidence$5,
                                 scala.reflect.api.TypeTags.TypeTag<A2> evidence$6)
```
    Defines a user-defined function of 2 arguments as user-defined function (UDF). The data types are automatically inferred based on the function's signature.
    
    Parameters:
    f - (undocumented)
    evidence$4 - (undocumented)
    evidence$5 - (undocumented)
    evidence$6 - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - udf
```
public static <RT,A1,A2,A3> UserDefinedFunction udf(scala.Function3<A1,A2,A3,RT> f,
                                    scala.reflect.api.TypeTags.TypeTag<RT> evidence$7,
                                    scala.reflect.api.TypeTags.TypeTag<A1> evidence$8,
                                    scala.reflect.api.TypeTags.TypeTag<A2> evidence$9,
                                    scala.reflect.api.TypeTags.TypeTag<A3> evidence$10)
```
    Defines a user-defined function of 3 arguments as user-defined function (UDF). The data types are automatically inferred based on the function's signature.
    
    Parameters:
    f - (undocumented)
    evidence$7 - (undocumented)
    evidence$8 - (undocumented)
    evidence$9 - (undocumented)
    evidence$10 - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - udf
```
public static <RT,A1,A2,A3,A4> UserDefinedFunction udf(scala.Function4<A1,A2,A3,A4,RT> f,
                                       scala.reflect.api.TypeTags.TypeTag<RT> evidence$11,
                                       scala.reflect.api.TypeTags.TypeTag<A1> evidence$12,
                                       scala.reflect.api.TypeTags.TypeTag<A2> evidence$13,
                                       scala.reflect.api.TypeTags.TypeTag<A3> evidence$14,
                                       scala.reflect.api.TypeTags.TypeTag<A4> evidence$15)
```
    Defines a user-defined function of 4 arguments as user-defined function (UDF). The data types are automatically inferred based on the function's signature.
    
    Parameters:
    f - (undocumented)
    evidence$11 - (undocumented)
    evidence$12 - (undocumented)
    evidence$13 - (undocumented)
    evidence$14 - (undocumented)
    evidence$15 - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - udf
```
public static <RT,A1,A2,A3,A4,A5> UserDefinedFunction udf(scala.Function5<A1,A2,A3,A4,A5,RT> f,
                                          scala.reflect.api.TypeTags.TypeTag<RT> evidence$16,
                                          scala.reflect.api.TypeTags.TypeTag<A1> evidence$17,
                                          scala.reflect.api.TypeTags.TypeTag<A2> evidence$18,
                                          scala.reflect.api.TypeTags.TypeTag<A3> evidence$19,
                                          scala.reflect.api.TypeTags.TypeTag<A4> evidence$20,
                                          scala.reflect.api.TypeTags.TypeTag<A5> evidence$21)
```
    Defines a user-defined function of 5 arguments as user-defined function (UDF). The data types are automatically inferred based on the function's signature.
    
    Parameters:
    f - (undocumented)
    evidence$16 - (undocumented)
    evidence$17 - (undocumented)
    evidence$18 - (undocumented)
    evidence$19 - (undocumented)
    evidence$20 - (undocumented)
    evidence$21 - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - udf
```
public static <RT,A1,A2,A3,A4,A5,A6> UserDefinedFunction udf(scala.Function6<A1,A2,A3,A4,A5,A6,RT> f,
                                             scala.reflect.api.TypeTags.TypeTag<RT> evidence$22,
                                             scala.reflect.api.TypeTags.TypeTag<A1> evidence$23,
                                             scala.reflect.api.TypeTags.TypeTag<A2> evidence$24,
                                             scala.reflect.api.TypeTags.TypeTag<A3> evidence$25,
                                             scala.reflect.api.TypeTags.TypeTag<A4> evidence$26,
                                             scala.reflect.api.TypeTags.TypeTag<A5> evidence$27,
                                             scala.reflect.api.TypeTags.TypeTag<A6> evidence$28)
```
    Defines a user-defined function of 6 arguments as user-defined function (UDF). The data types are automatically inferred based on the function's signature.
    
    Parameters:
    f - (undocumented)
    evidence$22 - (undocumented)
    evidence$23 - (undocumented)
    evidence$24 - (undocumented)
    evidence$25 - (undocumented)
    evidence$26 - (undocumented)
    evidence$27 - (undocumented)
    evidence$28 - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - udf
```
public static <RT,A1,A2,A3,A4,A5,A6,A7> UserDefinedFunction udf(scala.Function7<A1,A2,A3,A4,A5,A6,A7,RT> f,
                                                scala.reflect.api.TypeTags.TypeTag<RT> evidence$29,
                                                scala.reflect.api.TypeTags.TypeTag<A1> evidence$30,
                                                scala.reflect.api.TypeTags.TypeTag<A2> evidence$31,
                                                scala.reflect.api.TypeTags.TypeTag<A3> evidence$32,
                                                scala.reflect.api.TypeTags.TypeTag<A4> evidence$33,
                                                scala.reflect.api.TypeTags.TypeTag<A5> evidence$34,
                                                scala.reflect.api.TypeTags.TypeTag<A6> evidence$35,
                                                scala.reflect.api.TypeTags.TypeTag<A7> evidence$36)
```
    Defines a user-defined function of 7 arguments as user-defined function (UDF). The data types are automatically inferred based on the function's signature.
    
    Parameters:
    f - (undocumented)
    evidence$29 - (undocumented)
    evidence$30 - (undocumented)
    evidence$31 - (undocumented)
    evidence$32 - (undocumented)
    evidence$33 - (undocumented)
    evidence$34 - (undocumented)
    evidence$35 - (undocumented)
    evidence$36 - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - udf
```
public static <RT,A1,A2,A3,A4,A5,A6,A7,A8> UserDefinedFunction udf(scala.Function8<A1,A2,A3,A4,A5,A6,A7,A8,RT> f,
                                                   scala.reflect.api.TypeTags.TypeTag<RT> evidence$37,
                                                   scala.reflect.api.TypeTags.TypeTag<A1> evidence$38,
                                                   scala.reflect.api.TypeTags.TypeTag<A2> evidence$39,
                                                   scala.reflect.api.TypeTags.TypeTag<A3> evidence$40,
                                                   scala.reflect.api.TypeTags.TypeTag<A4> evidence$41,
                                                   scala.reflect.api.TypeTags.TypeTag<A5> evidence$42,
                                                   scala.reflect.api.TypeTags.TypeTag<A6> evidence$43,
                                                   scala.reflect.api.TypeTags.TypeTag<A7> evidence$44,
                                                   scala.reflect.api.TypeTags.TypeTag<A8> evidence$45)
```
    Defines a user-defined function of 8 arguments as user-defined function (UDF). The data types are automatically inferred based on the function's signature.
    
    Parameters:
    f - (undocumented)
    evidence$37 - (undocumented)
    evidence$38 - (undocumented)
    evidence$39 - (undocumented)
    evidence$40 - (undocumented)
    evidence$41 - (undocumented)
    evidence$42 - (undocumented)
    evidence$43 - (undocumented)
    evidence$44 - (undocumented)
    evidence$45 - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - udf
```
public static <RT,A1,A2,A3,A4,A5,A6,A7,A8,A9> UserDefinedFunction udf(scala.Function9<A1,A2,A3,A4,A5,A6,A7,A8,A9,RT> f,
                                                      scala.reflect.api.TypeTags.TypeTag<RT> evidence$46,
                                                      scala.reflect.api.TypeTags.TypeTag<A1> evidence$47,
                                                      scala.reflect.api.TypeTags.TypeTag<A2> evidence$48,
                                                      scala.reflect.api.TypeTags.TypeTag<A3> evidence$49,
                                                      scala.reflect.api.TypeTags.TypeTag<A4> evidence$50,
                                                      scala.reflect.api.TypeTags.TypeTag<A5> evidence$51,
                                                      scala.reflect.api.TypeTags.TypeTag<A6> evidence$52,
                                                      scala.reflect.api.TypeTags.TypeTag<A7> evidence$53,
                                                      scala.reflect.api.TypeTags.TypeTag<A8> evidence$54,
                                                      scala.reflect.api.TypeTags.TypeTag<A9> evidence$55)
```
    Defines a user-defined function of 9 arguments as user-defined function (UDF). The data types are automatically inferred based on the function's signature.
    
    Parameters:
    f - (undocumented)
    evidence$46 - (undocumented)
    evidence$47 - (undocumented)
    evidence$48 - (undocumented)
    evidence$49 - (undocumented)
    evidence$50 - (undocumented)
    evidence$51 - (undocumented)
    evidence$52 - (undocumented)
    evidence$53 - (undocumented)
    evidence$54 - (undocumented)
    evidence$55 - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - udf
```
public static <RT,A1,A2,A3,A4,A5,A6,A7,A8,A9,A10> UserDefinedFunction udf(scala.Function10<A1,A2,A3,A4,A5,A6,A7,A8,A9,A10,RT> f,
                                                          scala.reflect.api.TypeTags.TypeTag<RT> evidence$56,
                                                          scala.reflect.api.TypeTags.TypeTag<A1> evidence$57,
                                                          scala.reflect.api.TypeTags.TypeTag<A2> evidence$58,
                                                          scala.reflect.api.TypeTags.TypeTag<A3> evidence$59,
                                                          scala.reflect.api.TypeTags.TypeTag<A4> evidence$60,
                                                          scala.reflect.api.TypeTags.TypeTag<A5> evidence$61,
                                                          scala.reflect.api.TypeTags.TypeTag<A6> evidence$62,
                                                          scala.reflect.api.TypeTags.TypeTag<A7> evidence$63,
                                                          scala.reflect.api.TypeTags.TypeTag<A8> evidence$64,
                                                          scala.reflect.api.TypeTags.TypeTag<A9> evidence$65,
                                                          scala.reflect.api.TypeTags.TypeTag<A10> evidence$66)
```
    Defines a user-defined function of 10 arguments as user-defined function (UDF). The data types are automatically inferred based on the function's signature.
    
    Parameters:
    f - (undocumented)
    evidence$56 - (undocumented)
    evidence$57 - (undocumented)
    evidence$58 - (undocumented)
    evidence$59 - (undocumented)
    evidence$60 - (undocumented)
    evidence$61 - (undocumented)
    evidence$62 - (undocumented)
    evidence$63 - (undocumented)
    evidence$64 - (undocumented)
    evidence$65 - (undocumented)
    evidence$66 - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.3.0
  - udf
```
public static UserDefinedFunction udf(java.lang.Object f,
                      DataType dataType)
```
    Defines a user-defined function (UDF) using a Scala closure. For this variant, the caller must specify the output data type, and there is no automatic input type coercion.
    
    Parameters:
    f - A closure in Scala
    dataType - The output data type of the UDF
    
    Returns:
    (undocumented)
    Since:
    
    2.0.0
  - callUDF
```
public static Column callUDF(java.lang.String udfName,
             scala.collection.Seq<Column> cols)
```
    Call an user-defined function. Example:
```
  import org.apache.spark.sql._

  val df = Seq(("id1", 1), ("id2", 4), ("id3", 5)).toDF("id", "value")
  val spark = df.sparkSession
  spark.udf.register("simpleUDF", (v: Int) => v * v)
  df.select($"id", callUDF("simpleUDF", $"value"))
 
```
    Parameters:
    udfName - (undocumented)
    cols - (undocumented)
    
    Returns:
    (undocumented)
    Since:
    
    1.5.0

Class functions

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Constructor Detail

functions

Method Detail

countDistinct

countDistinct

array

array

map

coalesce

struct

struct

greatest

greatest

least

least

hash

concat

concat_ws

format_string

json_tuple

callUDF

col

column

lit

asc

desc

approxCountDistinct

approxCountDistinct

approxCountDistinct

approxCountDistinct

avg

avg

collect_list

collect_list

collect_set

collect_set

corr

corr

count

count

countDistinct

countDistinct

covar_pop

covar_pop

covar_samp

covar_samp

first

first

first

first

grouping

grouping

grouping_id

grouping_id

kurtosis

kurtosis

last

last

last

last

max

max

mean

mean

min

min

skewness

skewness

stddev

stddev

stddev_samp

stddev_samp

stddev_pop

stddev_pop

sum

sum