web-dev-qa-db-ja.com

AttributeError： 'module'オブジェクトに属性 'ensure_str'がありません

Beamを介して1つのbigqueryからantherにデータを転送しようとすると、次のエラーが発生します。

WARNING:root:Retry with exponential backoff: waiting for 4.12307941111 seconds before retrying get_query_location because we caught exception: AttributeError: 'module' object has no attribute 'ensure_str'
 Traceback for above exception (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/Apache_beam/utils/retry.py", line 197, in wrapper
    return fun(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/Apache_beam/io/gcp/bigquery_tools.py", line 261, in get_query_location
    response = self.client.jobs.Insert(request)
  File "/usr/local/lib/python2.7/site-packages/Apache_beam/io/gcp/internal/clients/bigquery/bigquery_v2_client.py", line 342, in Insert
    upload=upload, upload_config=upload_config)
  File "/usr/local/lib/python2.7/site-packages/apitools/base/py/base_api.py", line 703, in _RunMethod
    download)
  File "/usr/local/lib/python2.7/site-packages/apitools/base/py/base_api.py", line 674, in PrepareHttpRequest
    method_config.query_params, request, global_params)
  File "/usr/local/lib/python2.7/site-packages/apitools/base/py/base_api.py", line 551, in __ConstructQueryParams
    global_params, self.__client.global_params)
  File "/usr/local/lib/python2.7/site-packages/apitools/base/py/base_api.py", line 357, in global_params
    return encoding.CopyProtoMessage(self._default_global_params)
  File "/usr/local/lib/python2.7/site-packages/apitools/base/py/encoding_helper.py", line 112, in CopyProtoMessage
    return JsonToMessage(type(message), MessageToJson(message))
  File "/usr/local/lib/python2.7/site-packages/apitools/base/py/encoding_helper.py", line 123, in JsonToMessage
    return _ProtoJsonApiTools.Get().decode_message(message_type, message)
  File "/usr/local/lib/python2.7/site-packages/apitools/base/py/encoding_helper.py", line 309, in decode_message
    message_type, result)
  File "/usr/local/lib/python2.7/site-packages/apitools/base/protorpclite/protojson.py", line 209, in decode_message
    encoded_message = six.ensure_str(encoded_message)

これが私のコードです：

class SplitBDoFn(beam.DoFn):
  Word_tag = 'Word_tag'
  def process(self, element):
    if element:
      yield pvalue.TaggedOutput(self.Word_tag, element)

def run(argv=None):
  parser = argparse.ArgumentParser()
  known_args, pipeline_args = parser.parse_known_args(argv)
  pipeline_args.extend([
      '--runner=DirectRunner', 
      '--project=myproject',
      '--gcs_location=US',
      '--staging_location=gs://test-bucket/stage',
      '--temp_location=gs://test-bucket/temp',
      '--job_name=test-job',
  ])

  pipeline_options = PipelineOptions(pipeline_args)
  pipeline_options.view_as(SetupOptions).save_main_session = True
  pipeline_options.view_as(StandardOptions).streaming = True
  with beam.Pipeline(options = pipeline_options) as p:
    bq_source = beam.io.BigQuerySource(query = 'select * from myproject:raw_data.events where utc_date = "2019-07-20"')
    bq_data = p | beam.io.Read(bq_source)

    multiple_lines = (
        bq_data
        | 'SplitBDoFn' >> (beam.ParDo(SplitBDoFn()).with_outputs(
                                      SplitBDoFn.Word_tag)))

    Word_tag = multiple_lines.Word_tag

    (Word_tag
        | "output_Word_tag" >> beam.io.WriteToBigQuery(
                              table = 'test',
                              dataset = 'temp',
                              project = 'myproject',
                              schema = data_schema,
                              # validate = True,
                              write_disposition = beam.io.BigQueryDisposition.WRITE_APPEND,
                              create_disposition = beam.io.BigQueryDisposition.CREATE_IF_NEEDED
                        ))

ビームバージョン：2.13.0

誰かが以前にこの問題に遭遇できましたか？または私のコードに間違いはありますか？

pythongoogle-cloud-dataflowapache-beam
3
zangw

それは、1.12.0バージョンで6にensure_strが追加されたようであり、それは apitools を介してプールされるべきです。

根本的な原因は、仮想環境に古いバージョンの6つ（1.11以前）がインストールされていることだと思います。パイプラインを再試行する前に新しいvirtualenvを作成してみるか、または quick-start example を実行できますか？

1
Alex Amato