1.   Workflow description

Before going into explanation and definitions of different branches, we want to provide a global description of the work flow first. The workflow involves not only different branches but also different roles of people, different roles of testing servers. I assume that most of the participant’s roles are very familiar to you ( in other context, they are called stakeholders ), like “developers“, “quality controller“, “customer“, “project manager“, “release manager“…etc. From start, developer branch and master branch are the same. They are created initially at the same point; in the case that there is already a live production server running, they are created directly by mirroring exactly the same code based on the live production server. Team developers branch off every new feature or bug fixing branch from the develop branch. Each bug fix ticket has its own branch following a standard naming convention; each feature or new evolution has its own branch also following a standard naming convention. Developer works on the task for a bug or an evolution inside corresponding dedicated branch. When developer feels work done after tested on the development machine, he or she merges the branch into the “staging / internal integrated test branch“; then he or she logs into the internal integrated test server to perform a self integration test, to verify that the fix or task still works after merged with other team mates‘ code. After past the developer self testing, the bug fixing or the develop task then is marked as “resolved“. Quality controller team members screens all the “resolved“ tickets to know how many and which tickets needs to be tested at any given time. To start a test for a ticket, quality controller merge the corresponding bug fixing or feature branch into the “Developer test / Customer UAT branch“. Then, Quality controller logs into the Customer UAT server and perform integration test and / or any other tests, like regression test. When a ticket is past the test, it is possible that customer will verify the tested task on the Customer UAT server, since the customer will have access for the UAT server. When a ticket is tested by quality team and confirmed by customer on the Customer UAT server, the ticket is marked as “confirmed“ ( means ready to release ). At the end of the sprint cycle, quality team retrieve a list of the “confirmed“ tickets. This can be served as a report for the project manager or customer to know what work is ready to be delivered from the last sprint cycle. Project Manager (PM), or Release Manager (RM), screens all the “confirmed“ tickets to know how many and which tickets are pending for the next release. To start move tickets into release, PM or RM merge the corresponding ticket in to the “develop branch“. These tickets can be merged one by one, before finally push to the remote “develop branch“. After all the release content have been merged into the develop branch, PM or RM then merge the develop branch into the “master branch“, result is a new code change on the master branch. Each commit on master branch is corresponding to one time release. Optionally, each commit on the master branch can be assigned a tag to describe at give point what the release does. For the hot fix, it is very similar process to the normal bug fixing or feature release. The only difference is that the hot fix branch is branch off the “master branch“; and optionally, under customer’s urgent request, after it has been confirmed by test, it can be merged into “master branch“ to do a special earlier hot fix release.

2.   Branch definitions and explanations

There are 7 different types of branches: feature branches, bug fixing branches, hot fixing branchers, Staging / Internal integrated test branch, Develop test / Customer UAT branch, Develop branch and Master branch.

• Feature / bug fixing branch – Responsibility: Developers These branches are the main working branches worked by the engineers. As described above, they have to be branched off from the “develop“ branch. In the standard Git process jargon, they are called “topic“ branch.In our practice, there is a lightly difference between feature branch and bug fixing branch: Each bug fixing branch is corresponding to one ticket in the issue log, whereas a feature branch may includes many ticekts in our development backlog. Feature branch may have longer life time. Some features, may need multiple sprint cycles to finish —- we have some evolutions span over several months includes different components and milestones. In the long feature, there maybe many tickets ( or user stories ); in this case, we no need to split them into different branches. An advantages is that when many people working on the same feature, they are no need to merge or pull their codes back and forth, they are in the same branch, just keep the branch updated ( pull and push in time ).A feature branch’s life comes to an end when this feature is released. Subsequent tickets ( change request and new found bugs ) of this feature will be treated as bug fixing branches.

  • Feature branch naming convention:

    feature/[small-change-request-id]-ticketTitle

    feature/featureName

    Feature Branch Naming

    **Bug fixng branch naming convention:**bugfix/ticketId-ticketTitle

    Bugfixing Branch Naming

  • Hot fixing branches - Responsibility: Developers The only purpose of a Hot fixing branch is to treat the emergency issue discovered on the live production server. So, a hot fixing branch has to be branched off from the “master branch“. Subsequent workflow of the hot fixing is exactly the same as feature / bug fixing branch.In the picture above, there is a dotted line from hot fixing branch to master branch. This illustrates that there is an optional step for merging the “validated“ ( tested and verfied by customer) hot fixing to master branch at earlier release date for urgent and critical issues. But no matter what, the hot fixing branch will need to be merged into “develop branch“ to ensure it is not missed during next release.

    • Hot fixing naming convention: hotfix/ticketId-ticketTitle

  • Staging / Internal integrated test branch - Responsibility: Developers As described above, this branch is for the team’s internal test. Customer usually don’t care about what happens on the internal integrated test server.Developers are responsible to manage this branch, make sure all the tickets will be merged into this branch and tested on the server.

  • Develop test / Customer UAT branch - Responsibility: Quality teamThis branch is the code based for the customer accessible UAT server. QC / QA test team members are in charge of this branch’s integrity. Each developer “resolved“ task should be merged into this branch and tested by Quality team and very likely by customer ( depends on project’s quality assurance procedure ). No task should be released without test confirmation. After test pasted, the ticket status will marked as “confirmed“, indicate it is ready to deploy to live.

  • Develop branch – Responsibility: Project Manager (PM), or Release Manager (RM) PM or RM will in charge for the develop branch. Upon release date, PM or RM will according to the list of the “ready-to-release“ tickets since last release to merge all of them into the develop branch.

  • Master branch – Responsibility : Responsibility: Project Manager (PM), or Release Manager (RM) Master branch is exact same as live production server, at lease after each successful deployment. PM or RM will make sure that the master branch’s integrity. Each code changes on the master branch means a change that needs to be deployed to the live server. It is also possible that PM or RM will tag certain release on master to indicate a reach of a milestone or a new version of the product.

  3.   What we achieved and further considerations

  1. What we achievedWe have used this branching model in a complex project for two months now, we are very happy about the results. Finally, we have a clear and reliable way to achieve following Project Management and Delivery objectives:
     • Clear responsibility for developers. Developers are clear which branches they are up to and they are confident while they are working, no other’s code will affect their working branch. Also, developers know they should self test the code in the correct environment, a plus is that this make conflicts realization earlier, reduce possible code conflicts and regressions later.
     • **Clear responsibility for quality team.**Test team members are no longer worried about if they tested code on the right branch, because only one branch is for them. And the test server and environment is built on this branch. Since customer can also perform test on the UAT server based on this branch, they can do it at any time, no technical hard dependency on our internal test team. This means for critical, urgent customer care task, customer can test it with our internal test team together upon our developers resolution of the task.What’s more, we have flexibility for the workload of the quality control. I know the text book way of doing scrum is that we define the scope of a sprint before hand. But, we also don’t like to be in the situation that the list of tasks within a sprint felt like “heavy handed“ from above to the team: we must to deliver these 5-6 stories plus these amount of bug fixings this sprint, this is defined before hand, so it is very complex to:1. add customer new changes. 2. to handle special situations that result task needs more time. 3. to “command“ test team that they must finish testing those 5 tickets within 2 hours. 4. to have 8 out of 10 tickets pending on customer verification on the last day—thus need declare a sprint failure or need push customer to do over time.Nonetheless, we still need clear definition of a sprint. But the philosophy is that: everyone do the best to achieve whatever we can achieve, and everyone can check and verify what we did; we can adapt unforeseen changes and special conditions; and we can have way to evaluate past work to improve for the future.
     • **Clear responsibility for Project Managers (PMs), or Release Managers (RMs)**PM or RM is in charge of the develop branch and master branch. Any others to make changes to the develop branch and master branch is prohibited and not possible in the system. This ensure the stability of the master branch thus the production code, and also peace of mind of PM that nobody except him can change it. It is very clearly to see which content are ready to release and very easier to get them: just merge the right branch into develop.
  2. Further considerations
     • Why we always just merge the corresponding ticket or feature branch? Is it not make sense to merge the UAT branch to develop once the UAT branch is stable?First, we don’t merge the UAT branch into develop because the UAT branch is not stable and we don’t know if it ever will be. The UAT branch might includes part of the tickets has been verified, and part of others just not got time by the customer to verify yet even maybe all of the code were actually OK. Also, if we need ensure the UAT branch to reach stable, it means the test team and very involved customer needs ensure to verify all the tickets that is merged into it before a sprint is end. As in our reality, it is not very realistic. And the benefit fo the current approach is well described in previous section.Second, technically, we can still “merge“ the verified only tickets in the UAT branch into the develop branch. But, this means use the git cherry pick command. For users come from SVN, merge in git is actually put all the history of a branch into another branch, you can’t actually merge just one commit ( unless the branch currently only has one commit ). “git cherry-pick“ command allows you to just put the changes for one commit into another branch, so technically this can be done. We have used cherry pick in other special case, but we don’t think it is a good idea to use “cherry pick“ as one of our normalized git process.
     • A Case of “rebase“. In standard git flow, the topic branches ( feature or bug fixing branches ) are not pushed on remote repository; thus it is said safe for developers to do rebase. In our experience, we pushed every branches into the remote repository part because customers requirements part because considering provide more flexibility for future adding “throw away“ VMs using tools like Docker (http://www.docker.com/). However, because of our workflow, it is safe to rebase some feature branches on develop branch from time to time. For long running features, during which the develop branch will go through different release cycles and updated many times; the developers needs make sure they are working on an code environment that is up-to-date and incorporated with latest stable code. This is critical to avoid big problems in later integration testing phase. Possibly, a new VM act as a very stable “preprod / production mirror“ based on the develop branch. As in the picture illustrate before, you probably realized that there is no process to verify the develop branch actually has working stable code. According to our experience so far, at this stage, the develop branch code base is stable enough, though it is a good idea to have another server just to act as a “product mirror“ like environment. Note, if we need move the customer verification step on to this new VM, it means the develop branch will have to be wait for confirmation of all tickets‘ code; thus we lost the knowledge that the develop branch always contain the “tested and verified“ code. If a new VM needs to be added based on the develop branch, we may have opportunity to consider to adjust the whole workflow to utilize this new VM more meaningfully.

← Back to all posts