我正在嘗試在 PySpark 中創建一個帶有陣列的資料框，如上所示，但它回傳推斷架構錯誤：
data = array([-1.01835623e-01, -2.81103030e-02, 9.39835608e-01, 1.45413309e-01,
3.11870694e-01, 4.00573969e-01, -2.64698595e-01, -4.19898927e-01,
-1.18507199e-01, -3.59607369e-01, 4.42910716e-02, 6.56066418e-01,
2.20986709e-01, -4.60361429e-02, -4.06525940e-01, -2.33521834e-01])
column = ['feature']
from pyspark.sql.types import StructType, StructField, LongType
schema = StructType([StructField("feature", LongType(), True)])
dataframe = spark.createDataFrame(data, column, schema)
dataframe.show()
**TypeError: Can not infer schema for type: <class 'numpy.float32'>**
我應該嘗試使用 NumPy 進行一些轉換還是有人有提示？
uj5u.com熱心網友回復：
這個 DoubleType 對我有用。
data = [('1',[-1.01835623e-01, -2.81103030e-02, 9.39835608e-01, 1.45413309e-01,
3.11870694e-01, 4.00573969e-01, -2.64698595e-01, -4.19898927e-01,
-1.18507199e-01, -3.59607369e-01, 4.42910716e-02, 6.56066418e-01,
2.20986709e-01, -4.60361429e-02, -4.06525940e-01, -2.33521834e-01])]
schema = StructType( [StructField("ID",StringType(),True),
StructField("feature",ArrayType(DoubleType()),True)])
df =spark.createDataFrame(data, schema)
--- --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|ID |feature |
--- --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|1 |[-0.101835623, -0.028110303, 0.939835608, 0.145413309, 0.311870694, 0.400573969, -0.264698595, -0.419898927, -0.118507199, -0.359607369, 0.0442910716, 0.656066418, 0.220986709, -0.0460361429, -0.40652594, -0.233521834]|
--- --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
轉載請註明出處，本文鏈接：https://www.uj5u.com/shujuku/507399.html