2021 – Page 44 – My Point of View

Azure DevOps Branching and Release Flow Guide For beginners’

Problem

I am working with a small team (2-4 developers). We are new to Azure DevOps. We would like to start simple with this technology.

We are supporting a code base that’s is working for a single office. Client would like to enhance the code base to support multiple offices. Client doesn’t want to re-write the whole application but would like to extend single office code base to support multiple offices.

Design

There are two challenges here;

The first challenge is that team is new to Azure DevOps. What could be the best simplistic approach to follow?

We decided to go with “Basic process” template. All work items can be grouped under Epic, Issues and Tasks. There is a simple kabana dashboard that show work progress.

For merging through pull request, we decided to go with “Merge (No Fast Forward)” method. This method will persist commits in staging branch and re-create those commits in main branch. This is verbose though.

We decided that there should be just two roles, Administrator and Developers. Administrator is responsible for release management and root level branches and Developers can create / merge feature and bug fixes.

For repo structure, we agreed to use this structure;

main <–> feature

main –> release

release <–> bugfix –> main

The second challenge is that client is not interested in code re-write for now. How do we keep a single repository and support two main branches?

We would go with a single repository, for example HOO. Ideally a single repository should have one main branch but to support our specific situation, we would create a separate long running branch from our main branch.

Since single office code base will be supported on regular basis so all new work will be merged into main default branch via pull request.

Multiple office code base branch can be worked on by creating branches from it. Those branches will be merged back into multiple code office branch. This approach will help us to rebase/merge/cherry pick features from main branch into multiple office branch.

For the sake of simplicity, we refer S as single branch and M as Multiple office branch.

Prerequisite

Azure DevOps Account
Git for windows

You can download from here

https://git-scm.com/download/win

Open Visual Studio, Check the Git for Windows in the Tools – Get Tools and Features…), go to “Individual Item” tab, check “Git for Windows”, and click “Modify”. Then it will ask you to update vs2017 to latest version, for example 15.9.36.

3. Single office code repo is created

If you don’t have a repository in a project, refer to this article;

Create repository and upload main branch

Solution

I am going to take very simple approach to demonstrate this flow from branching to release to bugfix and merging everything back into main.

I will be using a single about.cshtml page. I am going to make first commit by adding this line of code;

Democracy died in darkness

I will release this code to client by creating a branch from main.

While customer is working with my first relese, I will add another line of code and commit my changes to main.

Tell me and I forget. Teach me and I remember. Involve me and I learn

Customer feedback comes with a bugfix correction in my first release;

Democracy dies in darkness – Light up the darkness

So far, I have made two commits into main branch. I need to fix a bug, make a release and port my bug fix back into main. At the end of this exercise,

I need to make sure that first commit is replaced, second commit should stay in my main branch.
I need to make sure that release has code fixed and my second commit should not go with release;

# Task-1 – Create folder structure and assign permissions

Run the following commands in the Developer Command Prompt, under Start > Visual Studio > Developer Command Prompt. Each command is preceded with an explanation of what it’s doing. If you don’t have a personal access token cached (for example by signing in to the Azure DevOps Services web portal) you’ll be prompted to login.

First, block the Create Branch permission at the repository root for the project’s contributors.

tf git permission /deny:CreateBranch /group:[PMB]\Contributors /collection:https://dev.azure.com/{org name}/ /teamproject:PMB /repository:PMB

Then, allow contributors to create branches under feature, users, and bugfix.

tf git permission /allow:CreateBranch /group:[PMB]\Contributors /collection:https://dev.azure.com/{org name}/ /teamproject:PMB /repository:PMB /branch:feature

tf git permission /allow:CreateBranch /group:[PMB]\Contributors /collection:https://dev.azure.com/{org name}/ /teamproject:PMB /repository:PMB /branch:users

tf git permission /allow:CreateBranch /group:[PMB]\Contributors /collection:https://dev.azure.com/{org name}/ /teamproject:PMB /repository:PMB /branch:bugfix

tf git permission /allow:CreateBranch /group:[PMB]\Contributors /collection:https://dev.azure.com/{org name}/ /teamproject:PMB /repository:PMB /branch:hotfix

Allow administrators to create branches under release.

tf git permission /allow:CreateBranch /group:”[PMB]\Project Administrators” /collection:https://dev.azure.com/{org name}/ /teamproject:PMB /repository:PMB /branch:release

tf git permission /allow:CreateBranch /group:”[PMB]\Project Administrators” /collection:https://dev.azure.com/{org name}/ /teamproject:PMB /repository:PMB /branch:deploy

Finally, allow administrators to create a branch called main / master /stage (in case it ever gets accidentally deleted).

tf git permission /allow:CreateBranch /group:”[PMB]\Project Administrators” /collection:https://dev.azure.com/{org name}/ /teamproject:PMB /repository:PMB /branch:main

tf git permission /allow:CreateBranch /group:”[PMB]\Project Administrators” /collection:https://dev.azure.com/{org name}/ /teamproject:PMB /repository:PMB /branch:main-pmb

Administrator will create an initial release branch of Single office from main in release folder; This branch would be locked.

Administrator will create a branch policy so that all merges go through pull request. Since it’s a small team so requestor is allowed to approve its own request. This will be done through GUI.

Our final branch structure on root level is;

You can see an icon right in front of main, this shows branch policies are applied on our main line. You can also see a lock in front of our release branch. This will prevent code changes and commits.

All branches are on the same level with main. There are 0|0 commits behind and ahead.

# Task-2 – Clone / Start Over repository on developer machine

Go to you local drive and create a folder for example, C:\Dev
git clone <repo link from Azure DevOps>

This will clone your repo in C:\Dev folder. By default, it will get code from default branch, main. Switch to your repo directory.

git fetch

This will get latest code from the branch

git status

This will show you what branch you are on and the status of your branch. Usually this would be your default main branch.

# Task-3 – Development flow – Create first feature (Role = Developer)

The developer is going to add first line of code by creating a feature branch to work;

Go to you local drive folder, C:\Dev

This is where you have cloned your repository in Task #2.

git checkout master
git pull
git status

This will show you the last branch you were on.

git branch feature/S-F1

This will create a new branch on your local.

git checkout feature/S-F1

switch to new branch

git branch

git fetch

get latest from remote. Though this branch is not pushed to remote but it’s a good practice to use this command.

Open visual studio. If you don’t see your branch in the lower right corner then click and select feature branch. Add a paragraph to About.cshtml file.

Democracy died in darkness

run application. If everything looks good, save all and return to the command prompt.

git status

this will show you the status of your branch and changes;

git add .

This will add the file name that has changes into it.

git commit -m”Starting simple with Azure DevOps and Git added”

Add a description for your commit.

git push origin feature/S-F1

This will push our local branch to remote branch.

view your local commits that has not been pushed to remote

git log origin/master..HEAD

you can also view the diff of your local stagged and remote branch using this;

git diff origin/main..HEAD

if you want to see all the changes you made in a feature branch before submitting a PR, using command line you can do this:

git checkout main

git merge <branch-name> –no-commit –no-ff

git merge feature/s-reset –no-commit –no-ff

Look at Changes in Team Explorer to see all the changes you made in a branch before submitting a PR.

Click on Abort. main branch is not affected by your changes.

If you look at Azure DevOps UI, you will see that this branch is 1 commit ahead of main branch.

“Create a pull request” flow

Fill up the form that shows up. Remember to add reviewer and add yourself. click Create button;

You will see this next page once create is completed;

Move optional reviewer to Required. Click on Approve. After approve, you will see complete button right beside it. Click on complete.

Uncheck “Delete feature/branch name” after merging to preserve branch.

Click on Pull Requests and Completed link. You will see this;

Merge is complete. This would be the complete page;

Switch to all branches, this is would be your branches view;

We can see that feature/S-F1 branch is 1 commit behind main. The commit is pull request commit that went into master. It’s ok.

Switch back to command prompt to refresh your local main branch;

git checkout main
git pull

Developer feature work is merged successfully in main branch.

# Task-4 – Release flow – Create first release (Role = Administrator)

This is an administrative job. So, administrator will use UI and create a release branch off of main branch.

Put a lock on this branch. Hand this release over to change management.

This would be your branch view at this stage;

# Task-5 – Development flow – add second code line (Role = Developer)

The steps are exactly the same as Task-3. We are going to add our second code line.

A standard practice is to create a new branch off of main branch to add new work. For the sake of demo, I am going to use same branch for new work.

Follow these git commands;

git checkout feature/S-F1
git pull
git status
Open visual studio. If you don’t see your branch in the lower right corner then click and select feature branch. Add a paragraph to About.cshtml file.

Tell me and I forget. Teach me and I remember. Involve me and I learn

run application. If everything looks good, save all and return to the command prompt.

git status

this will show you the status of your branch and changes;

git add .

This will add the file name that has changes into it.

git commit -m”second line of code added”

Add a description for your commit.

git push origin feature/S-F1

This will push our local branch to remote branch.

go to devops branches and go through pull request sequences.

Create, Approve and complete pull request as we did in Task #3. These changes will be merged into main branch.

This is how our branches will look like;

S-F1branch is two commit behind main branch. These two commits are PR commits. That’s fine.

git checkout main

git pull

# Task-6 – Bugfix flow – Need to fix first line of code (Role = Administrator and Developer)

Administrator will create a bugfix branch from release branch.

Developer will checkout this branch and start working;

git pull

get remote branch

git checkout bugfix/S-R2.0-F1
Open visual studio. If you don’t see your branch in the lower right corner then click and select feature branch. Add a paragraph to About.cshtml file.

Democracy dies in darkness – Light up the darkness

run application. If everything looks good, save all and return to the command prompt.

git status

this will show you the status of your branch and changes;

git add .

This will add the file name that has changes into it.

git commit -m”bug fixed in code line 1”

Add a description for your commit.

git push origin bufix/S-R2.0-F1

This will push our local branch to remote branch.

go to devops branches and go through pull request sequences.

Make sure you are merging it into your target release branch not in main branch.

Create, Approve and complete pull request as we did in Task #3. These changes will be merged into release branch.

check you release branch

git checkout release/S-R2.0-F1

git pull

I opened About.cshtml page and can see the line is fixed. Send “release/S-R2.9-F1” out to customer.

It’s time to merge this change in main branch. follow these steps;

git checkout main

git pull

git status

git branch feature/bugfix-S-R2.0-F1

create a temporary branch from main branch to merge changes from

bugfix/S-R2.0-F1 branch.

git checkout feature/bugfix-S-R2.0-F1

git push origin feature/bugfix-S-R2.0-F1

push this branch to remote. We don’t need to make any changes to it. It’s temporary.

We wouldn’t go through Pull request here because it’s our internal function. Pull request doesn’t help to merge and resolve conflicts.

Go back to VS and merge your changes following these steps;

Click on manage branches, click on merge. You will see this view;

Click on Merge. If there are conflicts, you will see this view;

Click on conflicts and Compare Files;

For some strange reasons, my merging is not working right. Here is demo output;

Source view;

Target view;

If I take source changes, it overwrites line 23. Not right.

If I take target changes, bugfix change gets lost. Not right.

The work around is to click merge button, you will get a view like this;

Top left is source, right is target and bottom is the result view. Click on check box from left and right. You will see code from both sides in result view. Make changes here by deleting this line;

Democracy died in darkness

This would be your final view;

Click on Accept changes, Commit Merge, Commit staged and sync.

Create a pull request to merge feature/bugfix-firstline back into master.

git checkout main

git pull

the main branch has all changes and its up to date with developers work and bug fixes.

# Task-7 – Bugfix flow – Manually merge code into main (Role = Developer)

This is a manual option. You are already working on branch which I can call a working branch. Copy changes from your bugfix/S-R2.0-F1 branch into your working branch. Bugfix changes will be ported over to your main when you do a pull request.

This method doesn’t require creating additional branch from master just for merging bugfixes into main. The con is deviating from process flow and possibility of introducing additional bugs.

# Task-8 – create a branch for multi offices (Role = Administrator)

We will create a branch from our main branch for example “main-m”. This branch can be used as a long running branch for our all-offices work. Developer’s will create working branches out of this branch.

#9 – Key git command list

I prefer using Git bash command editor. Go to your working folder and open git bash command window through context. These are some of common git commands;

git clone <repo link from web site>

Cloned your repo in your desired folder.

git status

will show you which branch you are on, and any commits you have made locally

git branch

will show you your current branch

git log

will show all commits

git branch <branch name>

create a new branch for example git branch feature/feature1

git fetch

get remote branches to your local. Gets new data from a report repository – but it doesn’t integrate any of this new data into your working files. This one is used infrequently.

git pull

This will update your current HEAD branch with the latest changes from remote server. It will download new data and directly integrate into your current working copy. This one is used frequently.

git checkout <branch name>

switch to another branch name, for example git checkout feature/feature1

git add .

if you made a change in the code and you want to commit locally, use it

git commit -m”I hhave made a change”

add description to your commit

git push origin <branch name>

push your local changes to remote repo. For example git push origin feature/feature1

git reset –hard origin/<branch name>

git reset –hard origin/feature/bugfix-S-R2.0-F1

git fetch

If your commits are only visible to you, you can do this; this will move back to where the origin is. This will reset the state of the repository to the previous commit, and it will discard all local changes. It will reset the project configuration as well, so take care of this. This will reset .cfg file to default.

git branch -d <branch name>

delete your local branch

if you want to switch to another branch, and you started seeing that there are uncommitted changes here; simply run these commands;

git add .

git commit “unwanted commit”

now you will be able to switch branches.

git branch -D <branch name>

delete an unmerged branch from your local.

git push origin –delete <branch name>

delete your remote branch, for example “git push origin –delete feature/feature1. Remember, you will need the Git ‘ForcePush’ permission to perform this action. This permission can be granted through Project settings -> permission tab.

How to view changes before submitting a PR request?

Go to Team Explorer > Branches > Right click your branch > View History > click the latest commit > ctrl + click the latest commit from master > right click > Compare Commits

this only shows you the diff between commits.

if you want to see all the changes you made in a feature branch before submitting a PR, using command line you can do this:

git checkout master

git merge <branch-name> –no-commit –no-ff

and then look at Changes in Team Explorer to see all the changes you made in a branch before submitting a PR.

How to compare two branches for differences?

I couldn’t find an easier way. The old method is to copy the target branch on a different folder on your local. Run VSDiff utility. Source would be your git branch and target would be your other folder. You would be able to see all differences.

view your local commits that has not been pushed to remote

git log origin/master..HEAD

you can also view the diff of your local stagged and remote branch using this;

git diff origin/main..HEAD

If you want to see all commits on all branches that aren’t pushed yet, you might be looking for something like this:

git log –branches –not –remotes

And if you only want to see the most recent commit on each branch, and the branch names, this:

git log –branches –not –remotes –simplify-by-decoration –decorate –oneline

Related links

Create repository and upload main branch

References

https://docs.microsoft.com/en-us/azure/devops/learn/devops-at-microsoft/release-flow

https://docs.microsoft.com/en-us/azure/devops/repos/git/require-branch-folders?view=azure-devops&tabs=browser

https://docs.microsoft.com/en-us/azure/devops/repos/git/git-branching-guidance?view=azure-devops

https://stackoverflow.com/questions/2016901/viewing-unpushed-git-commits

https://dzone.com/articles/git-operations-with-visual-studio-part-2

https://git-scm.com/book/en/v2/Git-Branching-Basic-Branching-and-Merging

https://www.gitkraken.com/learn/git/branch

Thank you all for reading this.

Create repository and upload main branch

Follow these steps;

I am assuming that project already exists and we are adding a new repository. Got to Project Settings –> Repositories–> Create new repository.

In the Create a new repository dialog, verify that Git is the repo type and enter a name for your new repo. You can also add a README and create a .gitignore for the type of code you plan to manage in the repo. A README contains information about the code in your repo. The .gitignore file tells Git which types of files to ignore, such as temporary build files from your development environment.

We will see this page;

The simple method is to clone this repository to your computer. Follow this;

Goto C:\Dev
git clone <repo name>

you will get a warning that you have cloned an empty repository. That’s ok

Goto C:\Dev\FM.LedgerServiceAPI

This is the folder that clone command has created

Copy your code into this folder. You can use file explorer or command line like this;

copy .. \source folder\*.* .

git add .
git commit -m”initial code pushed”
git push origin main

this will push your code to the main on remote server

git log

Your main branch is ready.

Run an SSIS package from SSMS with Transact-SQL

This quickstart demonstrates how to use SQL Server Management Studio (SSMS) to connect to the SSIS Catalog database, and then use Transact-SQL statements to run an SSIS package stored in the SSIS Catalog.

SQL Server Management Studio is an integrated environment for managing any SQL infrastructure, from SQL Server to SQL Database. For more info about SSMS, see SQL Server Management Studio (SSMS).

Prerequisites

Before you start, make sure you have the latest version of SQL Server Management Studio (SSMS). To download SSMS, see Download SQL Server Management Studio (SSMS).

An Azure SQL Database server listens on port 1433. If you’re trying to connect to an Azure SQL Database server from within a corporate firewall, this port must be open in the corporate firewall for you to connect successfully.

Supported platforms

You can use the information in this quickstart to run an SSIS package on the following platforms:

SQL Server on Windows.
Azure SQL Database. For more info about deploying and running packages in Azure, see Lift and shift SQL Server Integration Services workloads to the cloud.

You cannot use the information in this quickstart to run an SSIS package on Linux. For more info about running packages on Linux, see Extract, transform, and load data on Linux with SSIS.

For Azure SQL Database, get the connection info

To run the package on Azure SQL Database, get the connection information you need to connect to the SSIS Catalog database (SSISDB). You need the fully qualified server name and login information in the procedures that follow.

Log in to the Azure portal.
Select SQL Databases from the left-hand menu, and then select the SSISDB database on the SQL databases page.
On the Overview page for your database, review the fully qualified server name. To see the Click to copy option, hover over the server name.
If you forget your Azure SQL Database server login information, navigate to the SQL Database server page to view the server admin name. You can reset the password if necessary.

Connect to the SSISDB database

Use SQL Server Management Studio to establish a connection to the SSIS Catalog on your Azure SQL Database server.

Open SQL Server Management Studio.
In the Connect to Server dialog box, enter the following information:TABLE 1SettingSuggested valueMore infoServer typeDatabase engineThis value is required.Server nameThe fully qualified server nameIf you’re connecting to an Azure SQL Database server, the name is in this format: <server_name>.database.windows.net.AuthenticationSQL Server AuthenticationWith SQL Server authentication, you can connect to SQL Server or to Azure SQL Database. If you’re connecting to an Azure SQL Database server, you can’t use Windows authentication.LoginThe server admin accountThis account is the account that you specified when you created the server.PasswordThe password for your server admin accountThis password is the password that you specified when you created the server.
Click Connect. The Object Explorer window opens in SSMS.
In Object Explorer, expand Integration Services Catalogs and then expand SSISDB to view the objects in the SSIS Catalog database.

Run a package

Run the following Transact-SQL code to run an SSIS package.

In SSMS, open a new query window and paste the following code. (This code is the code generated by the Script option in the Execute Package dialog box in SSMS.)
Update the parameter values in the catalog.create_execution stored procedure for your system.
Make sure that SSISDB is the current database.
Run the script.
In Object Explorer, refresh the contents of SSISDB if necessary and check for the project that you deployed.

SQLCopy

Declare @execution_id bigint
EXEC [SSISDB].[catalog].[create_execution] @package_name=N'Package.dtsx',
    @execution_id=@execution_id OUTPUT,
    @folder_name=N'Deployed Projects',
	  @project_name=N'Integration Services Project1',
  	@use32bitruntime=False,
	  @reference_id=Null
Select @execution_id
DECLARE @var0 smallint = 1
EXEC [SSISDB].[catalog].[set_execution_parameter_value] @execution_id,
    @object_type=50,
	  @parameter_name=N'LOGGING_LEVEL',
	  @parameter_value=@var0
EXEC [SSISDB].[catalog].[start_execution] @execution_id
GO

Reference

https://docs.microsoft.com/en-us/sql/integration-services/ssis-quickstart-run-tsql-ssms?view=sql-server-ver15

How to run ssis package in asp.net core application?

You can package the assembly into a nuget package, Create a Lib folder inside your solution to hold the nuget package, then, create a nuget.config file to set the package sources to include the Lib folder inside your solution.

The following links contains more details about creating nuget package and hosting it locally:

Alternate Method – 1

To run SSIS package you need below DLLs in the code

Microsoft.SqlServer.ManagedDTS.dll
Microsoft.SqlServer.PipelineHost.dll
Microsoft.SqlServer.DTSRuntimeWrap.dll
Microsoft.SqlServer.DTSPipelineWrap.dll

It is easy to add DLLs in MVC projects, however in asp.net core it needs to be in form of a Nuget package.

So nuget package can be easily created using nuget package explorer. Below is the link

https://docs.nuget.org/create/using-a-gui-to-build-packages

In the nuget package explorer add a lib folder, inside that add a .net folder dnxcore50 and add the above DLLs. Click on tools analyse package and save the nuget.

In the visual studio 2015 solution, you can refer local packages. Tools – Nuget Package Manager – Package Manager Settings – Package source add the local package path.

After which you will be able to add the nuget package using nuget package manager and select local package as source

"dependencies": {
"Microsoft.AspNet.Hosting": "1.0.0-rc1-final",
"Microsoft.NETCore.Portable.Compatibility": "1.0.1-rc2-24027",
"SSISPackage": "1.0.0"
 }

"frameworks": {
"netcoreapp1.0": {
  "imports": [
    "dotnet5.6",
    "portable-net45+win8",
    "dnxcore"
  ]
  }
}

After which you will be able to use code to run SSIS package similar to MVC projects.

Application app = new Application();
        Package package = null;
        try
        {
            package = app.LoadPackage(@"C:\Files\Package.dtsx", null);
            Variables vars = package.Variables;
            vars["status"].Value = "ACTIVE";

            DTSExecResult results = package.Execute();

        }
        catch
        {
            return false;

        }
        finally
        {
            package.Dispose();
            package = null;
        }

Alternate Method – 2

You may now reference the dlls ( one from each) directly in your .net core project from the below locations to run ssis packages now

C:\Windows\Microsoft.NET\assembly\GAC_MSIL\Microsoft.SqlServer.ManagedDTS
C:\Windows\Microsoft.NET\assembly\GAC_MSIL\Microsoft.SqlServer.PipelineHost
C:\Windows\Microsoft.NET\assembly\GAC_64\Microsoft.SqlServer.DTSRuntimeWrap
C:\Windows\Microsoft.NET\assembly\GAC_MSIL\Microsoft.SqlServer.DTSPipelineWrap

you no longer need to create a nuget package

Reference

https://stackoverflow.com/questions/38791987/how-to-run-ssis-package-in-asp-net-core-application

Edit script in SSIS is not working

This started today. The edit script stopped working in VS 2017. Here is the work around;

The issue may occur because the Script Task is not installed properly.

VS 2017 solution

Here are two options that might help;

Option # 1

Open control panel and try to repair “Microsoft Visual Studio Tools for Applications 2017”.

This options has helped me after upgrading VS SSDT 2017 recently. The funny things is that you don’t need to repair “Microsoft SQL Server Data Tools for Visual Studio 2017 (SSDT). Anyway it worked.

Option # 2

Please check whether the Microsoft.SqlServer.ScriptTask.dll file exists in the following folders (supposing it is a 64-bit platform):

C:\Program Files\Microsoft SQL Server\110\DTS\Tasks
C:\Program Files (x86)\Microsoft SQL Server\110\DTS\Tasks
C:\Windows\Microsoft.NET\assembly\GAC_MSIL\Microsoft.SqlServer.ScriptTask\v4.0_11.0.0.0__89845dcd8080cc91

If the Microsoft.SqlServer.ScriptTask.dll file is missing in any of the folders, copy one (from the other folders or a different working SSIS server) to the folder. Alternatively, you can also use the command line tool gacutil.exe to install the task assembly into the global assembly cache (GAC)

If above work around does not work, then do this;

Uninstall SSDT from your machine

Reboot the machine so that no cache issue

Reinstall SSDT with the option Install new SQL Server Data Tools for Visual Studio 2017 instance (SSDT), like below. Do not install in your existing Visual Studio instance again.

VS 2019 Solution

Download and Install this patch from Microsoft;

https://www.microsoft.com/en-us/download/details.aspx?id=58317

Here is the reference to this download;

https://docs.microsoft.com/en-us/answers/questions/303968/visual-studio-2019-1690-ssis-script-task-vsta-wont.html

Hope this will help.