web-dev-qa-db-ja.com

列全体の大文字小文字を小文字に変更する方法は？

Spark Datasetで列全体の大文字小文字を小文字に変更したい

        Desired Input
        +------+--------------------+
        |ItemID|       Category name|
        +------+--------------------+
        |   ABC|BRUSH & BROOM HAN...|
        |   XYZ|WHEEL BRUSH PARTS...|
        +------+--------------------+

        Desired Output
        +------+--------------------+
        |ItemID|       Category name|
        +------+--------------------+
        |   ABC|brush & broom han...|
        |   XYZ|wheel brush parts...|
        +------+--------------------+

CollectAsList（）とtoString（）を試してみました。これは、非常に大規模なデータセットには時間がかかり、複雑な手順です。

また、「より低い」メソッドを見つけましたが、それをdasasetで動作させる方法を知りませんでした。上記を行う簡単で効果的な方法を提案してください。前もって感謝します

apache-sparkapache-spark-sqlspark-dataframeapache-spark-dataset
13
Shreeharsha

わかりました（Functions#lowerを使用、 Javadoc を参照）

import org.Apache.spark.sql.functions.lower

        String columnName="Category name";
        src=src.withColumn(columnName, lower(col(columnName)));
        src.show();

これにより、古い列がデータセット全体を保持する新しい列に置き換えられました。

        +------+--------------------+
        |ItemID|       Category name|
        +------+--------------------+
        |   ABC|brush & broom han...|
        |   XYZ|wheel brush parts...|
        +------+--------------------+
20
Shreeharsha

org.Apache.spark.sql.functionsのlower関数を使用します

例えば：

df.select($"q1Content", lower($"q1Content")).show

出力。

+--------------------+--------------------+
|           q1Content|    lower(q1Content)|
+--------------------+--------------------+
|What is the step ...|what is the step ...|
|What is the story...|what is the story...|
|How can I increas...|how can i increas...|
|Why am I mentally...|why am i mentally...|
|Which one dissolv...|which one dissolv...|
|Astrology: I am a...|astrology: i am a...|
| Should I buy tiago?| should i buy tiago?|
|How can I be a go...|how can i be a go...|
|When do you use  ...|when do you use  ...|
|Motorola (company...|Motorola (company...|
|Method to find se...|method to find se...|
|How do I read and...|how do i read and...|
|What can make Phy...|what can make phy...|
|What was your fir...|what was your fir...|
|What are the laws...|what are the laws...|
|What would a Trum...|what would a trum...|
|What does manipul...|what does manipul...|
|Why do girls want...|why do girls want...|
|Why are so many Q...|why are so many q...|
|Which is the best...|which is the best...|
+--------------------+--------------------+
14
Alberto Bonsanto