Supported DataStream Operators and UDFs
This section describes the DataStream operators and UDFs supported when using the OmniStream feature.
- The DataStream operators and UDFs described in this section are used for Flink 1.16.3 only.
- The UDF trustlist is provided from multiple dimensions, including data transfer objects, function types, UDF dependency classes and interfaces, Java type translation, and Java statement translation.
- If you use DataStream operators and UDFs that are not supported by OmniStream, the execution plan will be rolled back to native execution, which deteriorates the performance.
Supported DataStream Operators
OmniStream supports the following DataStream operators:
- Kafka Source
- Kafka Sink
- Map
- Reduce
- FlatMap
- Filter
UDF Trustlist
Supported data transfer objects:
- Long
- String
- Tuple2<String, Long>
Table 1 lists the supported dependency classes and interfaces. For details about other restrictions, see the UDF Translation Tool User Guide. The supported expressions may vary depending on the environment configuration. If you have any questions, contact local Huawei technical support.
Java Class |
Java Class Interface |
|---|---|
Arrays |
static <T> List<T> asList(Array) |
HashMap (The hashCode and equals methods must be implemented for all accessed elements.) |
Object get(Object key) |
Object put(Object key, Object value) |
|
void putAll(HashMap m) |
|
boolean containsKey(Object key) |
|
int size() |
|
bool remove (Object key) (Different from Java interfaces, variables cannot be used to carry return values.) |
|
Set<Map.Entry<Object,Object>> entrySet() |
|
Set<Object> keySet() |
|
HashMap clone() |
|
Iterator |
boolean hasNext() |
Object next() |
|
ArrayList |
Object get(int index) |
void clear() |
|
void add(Object e) |
|
Iterator iterator() |
|
boolean contains(Object o) |
|
int size() |
|
boolean isEmpty() |
|
LinkedList |
Object getFirst() |
Object getLast() |
|
void addLast(Object e) |
|
void addFirst(Object e) |
|
Map.Entry (The hash and equals methods must be implemented for elements in mapentry.) |
Object getKey() |
Object getValue() |
|
void setValue(Object value) (Different from Java interfaces, variables cannot be used to carry return values.) |
|
HashSet (The hash and equals methods must be implemented for accessed elements.) |
boolean addAll(ArrayList list) |
boolean add(Object e) |
|
boolean remove(Object o) |
|
boolean contains(Object o) |
|
int size() |
|
void clear() |
|
Iterator iterator() |
|
StringBuilder |
StringBuilder append(String str) |
String toString() |
|
Array (Only one-dimensional arrays of the object type are supported. Basic arrays and multi-dimensional arrays are not supported.) |
Size |
Get elements |
|
Put elements (in sequence) |
|
Integer |
String toString() |
bool equals(Integer *obj) override |
|
int intValue() |
|
static Integer valueOf(String s) |
|
static Integer valueOf(int i) |
|
Boolean |
static Boolean valueOf(boolean b) |
boolean booleanValue() |
|
Long |
int hashCode() |
boolean equals(Long obj) |
|
String toString() |
|
Long *clone() |
|
long longValue() |
|
static Long valueOf(String s) |
|
static Long valueOf(long l) |
|
Object |
int hashCode() |
bool equals(Object *obj) |
|
String toString() |
|
Object *clone() |
|
String |
int hashCode() |
boolean equals(String anObject) |
|
String toString() |
|
Object *clone() |
|
String replace(String target, String replacement) |
|
Array split(String regex) (Character strings can be split. Regular expressions are not supported.) |
|
String replaceAll(String regex, String replacement) |
|
int lastIndexOf(String str) |
|
int length() |
|
String substring(int beginIndex) |
|
String substring(int beginIndex, int endIndex) |
|
boolean contains(String s) |
|
boolean endsWith(String suffix) |
|
boolean startsWith(String prefix) |
|
Gson |
String toJson(HashMap<String,String> map) |
Map fromJson(String json, Type typeOf) (Only the String to Map type conversion is supported.) |
|
JsonObject |
JsonObject getAsJsonObject(String memberName) (Only String constants are supported.) |
JsonParser |
static JsonObject parseString(String json) |
JsonPrimitive |
boolean getAsBoolean() |
JsonElement |
JsonObject getAsJsonObject() |
double getAsDouble() |
|
float getAsFloat() |
|
int getAsInt() |
|
long getAsLong() |
|
short getAsShort() |
|
boolean getAsBoolean() |
|
String getAsString() |
|
boolean isJsonNull() |
|
String toString() |
|
String toString() |
|
JsonArray |
Iterator<JsonElement> iterator() |