Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error:kafka-connect-hdfs for MapR(maprfs) #270

Open
srinivas2494 opened this issue Jan 4, 2018 · 1 comment
Open

Error:kafka-connect-hdfs for MapR(maprfs) #270

srinivas2494 opened this issue Jan 4, 2018 · 1 comment

Comments

@srinivas2494
Copy link

srinivas2494 commented Jan 4, 2018

Hi all,

We are trying to stream data from kafka topics to maprfs. The processes we have tried

1)Installing MapR client on kafka server
2)Adding MapR jars along with kafka-connect-hdfs jars
3)changing hdfs to maprfs in properties file

command used to initiate kafka-connect
MAPR_TICKETFILE_LOCATION=/tmp/maprticket_XXXXX MAPR_HOME=/opt/mapr /bin/connect-standalone /etc/schema-registry/connect-avro-standalone.properties /etc/kafka-connect-hdfs/quickstart-hdfs.properties

These are MapR jars which are loading along with kafka-connect-jars
mapr jars

properties file:
name=maprfs-sink-connector
connector.class=io.confluent.connect.hdfs.HdfsSinkConnector

hadoop.conf.dir=/opt/mapr

tasks.max=1
topics=demo1-TEST
hdfs.url=maprfs:///
flush.size=1
logs.dir=/mapr/users/logs
topics.dir=/mapr/users/topics

Error we are getting; No file system for scheme:maprfs
screen shot 2017-12-11 at 11 40 05 pm

we are using
Confluentv4.0
MapR v5.2.1

@GeryD
Copy link

GeryD commented Nov 2, 2018

Old issue, you probably found the answer : you have to use mapr version of kafka connect.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants