Apache Camel sucks!

I was thinking on this post for some longer time, wondering if it’s Camel’s fault or if it’s my fault. After 8 months of working with Apache Camel my judgement is clear:

Apache Camel sucks!

DISCLAIMER: my opinions here are result of working with Apache Camel (versions 2.17 – 2.21) in context of integration with SFTP.

Apache Camel is designed such a way that every article/post comparing this framework to other integration frameworks (like Spring Integration) makes you think Apache Camel is simpler/easier (better?). Unfortunately this is only the first impression. It looks like a brilliant tool (with plenty of integration channels) however in reality it’s hard to configure and the documentation is not clear enough.

After 8 months of maintaining and extending a project using Apache Camel mainly for exchange of files through SFTP I think that Apache Camel is a significant source of problems.

  1. Two teams (from 2 countries) were working on configuring so called “routes” in Apache Camel and still these “routes” don’t fully work as expected.
  2. There were cases when Apache Camel was not logging processing errors (yes – logging or error handling was not configured as part of “route” definition) – still I think logging of unhandled exceptions is a must-have for such a framework.
  3. There were cases when a file was not created on SFTP server without any error or warning information provided by Apache Camel.
  4. It’s a real pain to configure Apache Camel to consume larger files exposed on SFTP server. I wanted the file to be deleted after consumption. However it looks that SFTP connection was lost during processing time (around 15 minutes). So the file stayed on SFTP server and Apache Camel started to consume it again (and again – endless). So I added “idempotent” option to avoid consuming files that were already processed. And it turned out it just doesn’t work as documented (and Apache Camel was still consuming the file again and again).
  5. Testing of Apache Camel “routes” is really hard (at least in context of SFTP integration).
  6. The last error (it convinced me finally to this blog post) I’ve observed was Apache Camel’s failure to generate a correct file on SFTP server. The file was consisting of multiple lines and collecting was done by Apache Camel’s (single lines were send to the Apache Camel bus). It turned out that Apache Camel overwritten parts of some randomly selected lines with other lines broking the output file format. What a shame! Restart of the application was needed to make it working again.

As a result the team lost a lot of time reading Apache Camel documentation and trying different configurations. Right now I strongly consider migrating to Spring Integration and removing Apache Camel wherever possible.

Conclusion

Stay out of Apache Camel as long as possible. If Apache Camel is present in your project because of some (like “political”) decision try to minimize the amount of responsibility in Apache Camel “routes”.

Advice

Avoid integration through SFTP (and FTP) at all costs. It’s an awful ancient protocol that should die and be abandoned forever. Every case of integration through SFTP can be replaced by using HTTP (HTTPS).

PS.

Some other problems with Apache Camel are nicely presented in this blog post: Alpakka (Akka Streams) vs Apache Camel: who wins?

About krzysztoftomaszewski

I've got M.Sc. in software engineering. I graduated in 2005 at Institute of Computer Science, Warsaw University of Technology, Faculty of Electronics and Information Technology. I'm working on computer software design and engineering continuously since 2004.
This entry was posted in Java, Spring and tagged , , . Bookmark the permalink.

11 Responses to Apache Camel sucks!

  1. Nnelson says:

    Hello Nrzysztoftomaszewski. Unfortunely I’ve read your post and im totally dissagre with that. The 6 points that you metion can be solved with a propertly configuration and the last part when you say as “Advice” makes me know that you really didn’t understand the documentation of Apache Camel. Well the purpose of this commend isn’t to judge your perspective about apache camel. I’ve been working with Apache like more than two years, and its the most easly and fast way to integrate and handle different services. I configured communication with the components TCP with netty4 as client and server, FTP, SFTP as client and server, many MQ, MyBatis, SQL, JMS, and obviusly that at the begining is really easy to create a communication between two services with a route. But when you need to do something more specific (probably is your case). Threre is where became your developer skills and use all components that camel provides you. Since i started used camel I implemented many applications using camel as a core of communication I didn’t had any problem with performance, data transfer, access to data, reading files and creates new one, , implementing quartz,etc. So, i guess that you must try simple tests using basic connections before implement bussiness logic.

    • 1) Can you point out some good source of hints to avoid such errors like Apache Camel overwriting part of one line with another line when generating a file on SFTP?

      2) My blog post was finally motivated by comparing 2 integrations with SFTP: one using Apache Camel and one using plain Jsch library. The second one caused almost no problems while we’ve seen many problems with the first one – based on Apache Came.

  2. Dmitry Shultz says:

    I think this post is some kind of joke, but i like to see if there are any related issues in the Camel tracking system. Did you open any? Let look…

    • No, it is not. Not at all. This is my opinion about Apache Camel based on my experience with this framework.

      BTW: The issue with Apache Camel generating broken CSV file happened again. So it happens like once a year. And one cannot reproduce it. Yeah, let’s create a ticket for this… so it is closed later with resolution like “Cannot reproduce”.

      • Dmitry Shultz says:

        I have several projects dealing with CSV’s back and forth running in 3 environments for more than 1 year. According to you I should hit the issue at least once.

        You missed the main point with Open Source communuty, if you have a ticket all can see exactly what you are talking about. Especislly If you can’t even reproduce it.

  3. Moreover I can add Apache Camel Kafka support is poor as well. Apache Camel provides artifact called “camel-kafka-starter”. It’s apparently something that should cooperate with Spring Boot framework. However this solution is really poor – it cannot make use of Spring properties for configuring Kafka client. All Kafka client parameters have to be explicitely given in every Apache Camel routes using Kafka. That’s unacceptable.

    • Dmitry Shultz says:

      I’m not using CDI, but anyway there is some big confusion or I’m missing something.
      I’m not sure why you need to configure Kafka client parameters anywhere else than in the client route. The whole point to use Camel client is to deal with the Camel compondnt/route config only.

      You have some weird requirements.

      PS, your comments functionality is the major pain in the ass. Where did the Google auth go after first comment?

  4. Now I’m working with applications using Apache Camel for more than 2 years and my opinion is the same. Our team takes every opportunity to get rid of this framework and we’re successful with this approach.

  5. Pez says:

    Camel is terrible and I will never participate on something like this. On the other side, good for programmers and their salary.

  6. Dan says:

    I’ve worked with Camel for about 5 years now. My experience has been similar. We have camel SFTP routes that put files on a remote server, and once every 6 months or so, it says ‘Cannot store file’, and the route just stays stuck with a .camelLock on that file, and stops processing, and the support team panics until i log in and delete the file and restart the route. Haven’t managed to work out from the docs or stackoverflow how to fix it.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s