"It's alright to follow the Apache Way, but finding My Way is essential as well." – Pan Juan

Apache ShardingSphere, a top Apache project and the first database middleware of the Apache Software Foundation, is gaining wider influence in China and abroad. In this article, we invited Ms. Pan Juan (Trista Pan), the core founding member of Apache ShardingSphere, Apache ShardingSphere PMC, and SphereEx co-founder & CTO, to introduce the open-sourcing journey of ShardingSphere.

The Apache Journey of ShardingSphere

It's not a strange name for techies concerned about global database applications. ShardingSphere is a flexible ecosystem of open source and distributed database solutions that is positioned as a Database Plus that aims at building a new ecosystem on top of a database that can provide enhanced services for databases. In our plans, ShardingSphere could give full play in this ecosystem, and we have been sparing no effort to make it come true.

ShardingSphere has been on the road of open source for six years and has gained over 15,000 stars on GitHub with more than 300 contributors globally.

The most significant milestone in ShardingSphere's open source journey, between 2018 and 2020, could be the entry into the Apache Software Foundation, and after a 17-month incubation period, it successfully graduated as a Top-Level Project. During these days, both ShardingSphere and I have learned a lot.

At first, it was difficult for us to donate ShardingSphere to the Apache Software Foundation, as there were not many mentors in China who were qualified to lead donations. We struggled for a long time until we met Mr. Jiang Ning and Mr. Wu Sheng from SkyWalking, who gave us a lot of guidance and advice to help ShardingSphere pass the vetting process. At that time I started to join the Apache Software Foundation.

After the project reached the incubator, the next step for us as "parents" is to raise the "baby", meaning you need to contribute codes and take the lead to build and operate the community simultaneously. It is vital to help the community promote its visibility and attract more partners, because only if we all do our bit can the project progress.

But community operation is much more complex than programming, and you should consider this more when you make decisions. Just like the story of the ShardingSphere community, because Zhang Liang, the other founder, and I had strong feelings for the project and wanted to do it right, we set strict standards about the quality of submitted codes at the beginning. But what we didn't realize was that this strictness prevented many newbies from contributing.

In that regard, we had to make a range of attempts and adjustments, including making clear guidelines, speeding up responses, putting some participation flow documents in the most prominent position, and even setting more simple tasks so that newbies can make the transition.

Today, after six years of continuous work, you can see that both the codes and the documentation of ShardingSphere are elegant and complete. Meanwhile, we strive to do some essential external jobs such as meetups and evangelism to attract more attention. In the early years, some developers in China might not have too much passion for open source projects because of copyright concerns and the uncertainty of payback. To reverse this situation, we can only do more evangelism to promote the open-source philosophy so that more technophiles can hold an objective view of it and show a willingness to participate in it.

In short, whether you are a programmer or a community administrator, you will always gain something meaningful from the process of enabling projects to grow. When I first got involved in open source projects, I was doing it out of interest, but then it became a habit. In this process, I could feel the improvement in my skills and vision, which was very worthwhile both for the project and for myself.

Later, when the project had developed more maturely, it was time to consider the "graduation". Just like in school, projects will face various tests as well. Is the community able to self-govern and develop sustainably? Have the copyrights, brands, and the project's release complied with laws and regulations during the incubation period? All of these kinds of questions need to be fully prepared to propose graduation from the foundation, or you have to make modifications and redo the whole process if you failed in the first place. Besides, after graduation, you still have to keep the community running and create value for everyone. That is how the open-source project ShardingSphere came to be today.

Commercialization Practices of SphereEx's

Since then, as ShardingSphere and the community continue to grow, more and more inspiring ideas have been injected into the project. However, surging requirements for R & D once stopped ShardingSphere from developing. In 2021, to break this barrier and further improve ShardingSphere, Zhang Liang and I founded SphereEx together. We wanted to leverage the power of commercialization to support the continued growth of ShardingSphere and add new momentum to the project through SphereEx.

At that time, ShardingSphere was mature enough, so we decided to open source all the underlying frameworks and features. As I am a huge fan of open source, and I benefit from it, I hope it can work the same for other peers in the community. For commercial customers, enterprise users with special needs are provided with some advanced features and complete solutions along with the core of ShardingSphere, which are two different operation models of open source and commercialized SphereEx.

Many founding members of SphereEx, including the peers we've recruited later, are generally from the ShardingSphere open source community. When I send invitations to new members, I will communicate with them frankly that if you are familiar with open source communities and have plans to become open source engineers, you are always welcome in SphereEx, and I'll pay you a full-time salary to do both open source and commercial projects. In this way, I hope more peers who are keen on open source will have the opportunity to devote themselves to doing more exciting and rewarding work.

I will also do my best to help peers around us through various approaches, such as suggesting R & D personnel write technical blogs, encouraging them to share at conferences or meetups, or involving operation colleagues to help technicians do editing and reviewing of their work.

SphereEx is quite different from those tech giants concerning the daily work, team members, and the overall atmosphere. We prefer to work on technology improvement rather than rush to the go-to-market. I do not think there are any problems if you spend the whole day without writing a single line of code, as long as you utilize your strengths in a meaningful way, such as thinking about a tech solution, writing an article, or sharing an experience.

In the future, SphereEx will continue to explore the commercialization of open source, and we will try to cooperate with cloud providers to migrate our products to the cloud and even overseas. Meanwhile, SphereEx will continue to assist in Apache ShardingSphere, and bring more community events and valuable meetups. I sincerely hope that the ShardingSphere community will become even better with the support of SphereEx.

From "Juan" to "Juan the Great"

Lastly, I'd like to share some personal experiences of my growth, leaving aside the titles of PMC, founder, and CTO. I used to be a DBA. It was a hard job to do and even harder to get promoted. This is why most newcomers, especially females, are unwilling to choose this position.

I knew that feeling when I was a DBA, which was the first time I realized what is called "suffering". It was in my early days when there was a migration project to be accomplished from 1 a.m. to 5 a.m., with no intervals. I probably didn't work well with colleagues in charge of the business, so the migration process wasn't smooth enough and needed to be rolled back, meaning all efforts were wasted. At that moment, I was devastated and couldn't hold on anymore as I hadn't slept for a whole day. At 3 a.m. I even burst into tears and said a lot of emotional words like, "I'm not doing this anymore. It makes no sense, no value at all. "

But business is business. You have to face the same task tomorrow and try to overcome the challenge ultimately. After that, I learned a very important lesson both for my personal growth and my career path: since I have determined to do this job at the very beginning, there shall be no excuses to escape from any obstacles. You should make complete preparations, face the difficulty, and carry on your tasks. In addition, I would always maintain a cautious attitude toward my work and double-check with my colleagues before implementing any operations, which enhances my ability to counter pressures and boosts my working competence.

In addition, I was the only female on the team, and everyone called me "Juan". After some time, my ability as a DBA was recognized, and my title was "escalated" to "Sister Juan," and then "Master Juan." Until one day, it became "Juan the Great." I don't care much about how people address me, and I think it's totally okay with all the nicknames because they're probably from friends or people around you to express respect and friendliness. But the changing titles could explain something: you are growing up step by step and doing better than before. Titles are meaningless, but I value the process and the experience more than the results.

In my starting-up journey, I have to do some evangelism and give speeches overseas. From my experience, I have found it hard to see Chinese faces in the list of 50 to 60 speakers, and sometimes I was the only one. Linguistic barriers, cross-disciplines, etc., may cause this situation. But as open-source projects are improving in China and our technical skills have been recognized overseas, we should do more and present our innovation to the whole world. Only in this way can we see our growth and push us to move forward. At the same time, technicians may also have the opportunity to unite and do even greater things together, which will form the force driving the whole industry.

Lastly, I sincerely hope that all techies and contributors to open source projects, as well as newcomers who are about to enter this industry, will share our desire to work together in the future. Then, we will be able to put our heads together and create more value for open source and the entire industry.

About Guest

Pan Juan (Trista Pan), SphereEx co-founder & CTO, Apache Member & Incubator mentor, Apache ShardingSphere PMC, AWS Data Hero, mentor of Mulan Open Source and Tencent TVP. Pan Juan was responsible for the design and development of the database intelligence platform of JD Digits, and now she focuses on open source and the ecosystem of distributed databases & middleware. She was named as the 2020 China Open Source Pioneer and the 2021 OSCAR Peak Open Source Figure.