Major revamp (#39)

* Major revamp * Fix action name * Use CMD instead of the action * fix path * Remove tty * Disabled tests * SPROCs removed from Reminders package. * SPROCs removed from persistence provider * Final tweaks * Update readme with migration paths.
OrleansContrib · Nov 14, 2019 · 280c792 · 280c792
1 parent 303c0f9
commit 280c792
Show file tree

Hide file tree

Showing 38 changed files with 1,684 additions and 1,608 deletions.
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -0,0 +1,26 @@
+name: CI
+
+on:
+  pull_request:
+    branches:
+      - master
+
+jobs:
+  build:
+
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v1
+    - name: Setup .NET Core
+      uses: actions/setup-dotnet@v1
+      with:
+        dotnet-version: 3.0.100
+    # - name: Azure Cosmos Emulator
+    #   uses: galvesribeiro/[email protected]
+    # - name: Start CosmosDB Emulator
+    #   run: .\StartEmulator.cmd
+    - name: Build
+      run: dotnet build --configuration Release
+    # - name: Test
+    #   run: dotnet test --configuration Release --no-build
diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml
@@ -0,0 +1,55 @@
+name: Publish
+
+on:
+  push:
+    tags:
+      - 'v*'
+
+jobs:
+  build:
+
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v1
+    - name: Setup .NET Core
+      uses: actions/setup-dotnet@v1
+      with:
+        dotnet-version: 3.0.100
+    - name: Pack
+      working-directory: src/Orleans.Clustering.CosmosDB
+      run: dotnet pack --configuration Release -p:Version=${GITHUB_REF##*/v}
+    - name: Pack
+      working-directory: src/Orleans.Persistence.CosmosDB
+      run: dotnet pack --configuration Release -p:Version=${GITHUB_REF##*/v}
+    - name: Pack
+      working-directory: src/Orleans.Reminders.CosmosDB
+      run: dotnet pack --configuration Release -p:Version=${GITHUB_REF##*/v}
+    - name: Pack
+      working-directory: src/Orleans.Streaming.CosmosDB
+      run: dotnet pack --configuration Release -p:Version=${GITHUB_REF##*/v}
+    - name: Push
+      working-directory: src/Orleans.Clustering.CosmosDB/bin/Release
+      run: |
+        dotnet nuget push Orleans.Clustering.CosmosDB.*.nupkg -k ${{ secrets.NUGET_KEY }} -s https://api.nuget.org/v3/index.json
+    - name: Push
+      working-directory: src/Orleans.Persistence.CosmosDB/bin/Release
+      run: |
+        dotnet nuget push Orleans.Persistence.CosmosDB.*.nupkg -k ${{ secrets.NUGET_KEY }} -s https://api.nuget.org/v3/index.json
+    - name: Push
+      working-directory: src/Orleans.Reminders.CosmosDB/bin/Release
+      run: |
+        dotnet nuget push Orleans.Reminders.CosmosDB.*.nupkg -k ${{ secrets.NUGET_KEY }} -s https://api.nuget.org/v3/index.json
+    - name: Push
+      working-directory: src/Orleans.Streaming.CosmosDB/bin/Release
+      run: |
+        dotnet nuget push Orleans.Streaming.CosmosDB.*.nupkg -k ${{ secrets.NUGET_KEY }} -s https://api.nuget.org/v3/index.json
+    - name: Create Release
+      uses: actions/create-release@master
+      env:
+        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      with:
+        tag_name: ${{ github.ref }}
+        release_name: Release ${{ github.ref }}
+        draft: false
+        prerelease: false
diff --git a/.vscode/launch.json b/.vscode/launch.json
@@ -0,0 +1,27 @@
+{
+   // Use IntelliSense to find out which attributes exist for C# debugging
+   // Use hover for the description of the existing attributes
+   // For further information visit https://github.com/OmniSharp/omnisharp-vscode/blob/master/debugger-launchjson.md
+   "version": "0.2.0",
+   "configurations": [
+        {
+            "name": ".NET Core Launch (console)",
+            "type": "coreclr",
+            "request": "launch",
+            "preLaunchTask": "build",
+            // If you have changed target frameworks, make sure to update the program path.
+            "program": "${workspaceFolder}/test/Orleans.CosmosDB.Tests/bin/Debug/netcoreapp3.0/Orleans.CosmosDB.Tests.dll",
+            "args": [],
+            "cwd": "${workspaceFolder}/test/Orleans.CosmosDB.Tests",
+            // For more information about the 'console' field, see https://aka.ms/VSCode-CS-LaunchJson-Console
+            "console": "internalConsole",
+            "stopAtEntry": false
+        },
+        {
+            "name": ".NET Core Attach",
+            "type": "coreclr",
+            "request": "attach",
+            "processId": "${command:pickProcess}"
+        }
+    ]
+}
diff --git a/.vscode/tasks.json b/.vscode/tasks.json
@@ -0,0 +1,42 @@
+{
+    "version": "2.0.0",
+    "tasks": [
+        {
+            "label": "build",
+            "command": "dotnet",
+            "type": "process",
+            "args": [
+                "build",
+                "${workspaceFolder}/Orleans.CosmosDB.sln",
+                "/property:GenerateFullPaths=true",
+                "/consoleloggerparameters:NoSummary"
+            ],
+            "problemMatcher": "$msCompile"
+        },
+        {
+            "label": "publish",
+            "command": "dotnet",
+            "type": "process",
+            "args": [
+                "publish",
+                "${workspaceFolder}/Orleans.CosmosDB.sln",
+                "/property:GenerateFullPaths=true",
+                "/consoleloggerparameters:NoSummary"
+            ],
+            "problemMatcher": "$msCompile"
+        },
+        {
+            "label": "watch",
+            "command": "dotnet",
+            "type": "process",
+            "args": [
+                "watch",
+                "run",
+                "${workspaceFolder}/Orleans.CosmosDB.sln",
+                "/property:GenerateFullPaths=true",
+                "/consoleloggerparameters:NoSummary"
+            ],
+            "problemMatcher": "$msCompile"
+        }
+    ]
+}
diff --git a/.vsts-ci.yml b/.vsts-ci.yml
diff --git a/.vsts-release.yml b/.vsts-release.yml
diff --git a/README.md b/README.md
@@ -3,10 +3,11 @@
   <h1>Orleans CosmosDB Providers</h1>
 </p>
 
-[![CircleCI](https://circleci.com/gh/OrleansContrib/Orleans.CosmosDB.svg?style=svg)](https://circleci.com/gh/OrleansContrib/Orleans.CosmosDB)
+[![CI](https://github.com/OrleansContrib/Orleans.CosmosDB/workflows/CI/badge.svg)](https://github.com/OrleansContrib/Orleans.CosmosDB/actions)
 [![NuGet](https://img.shields.io/nuget/v/Orleans.Clustering.CosmosDB.svg?style=flat)](http://www.nuget.org/packages/Orleans.Clustering.CosmosDB)
 [![NuGet](https://img.shields.io/nuget/v/Orleans.Persistence.CosmosDB.svg?style=flat)](http://www.nuget.org/packages/Orleans.Persistence.CosmosDB)
-[![NuGet](https://img.shields.io/nuget/v/Orleans.Reminders.CosmosDB.svg?style=flat)](http://www.nuget.org/packages/Orleans.Clustering.CosmosDB)
+[![NuGet](https://img.shields.io/nuget/v/Orleans.Reminders.CosmosDB.svg?style=flat)](http://www.nuget.org/packages/Orleans.Reminders.CosmosDB)
+[![NuGet](https://img.shields.io/nuget/v/Orleans.Streaming.CosmosDB.svg?style=flat)](http://www.nuget.org/packages/Orleans.Streaming.CosmosDB)
 [![Gitter](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/dotnet/orleans?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge)
 
 [Orleans](https://github.com/dotnet/orleans) is a framework that provides a straight-forward approach to building distributed high-scale computing applications, without the need to learn and apply complex concurrency or other scaling patterns. 
@@ -22,33 +23,33 @@ Installation is performed via [NuGet](https://www.nuget.org/packages?q=Orleans+C
 
 From Package Manager:
 
-> PS> Install-Package Orleans.Clustering.CosmosDB -prerelease
+> PS> Install-Package Orleans.Clustering.CosmosDB
 
-> PS> Install-Package Orleans.Persistence.CosmosDB -prerelease
+> PS> Install-Package Orleans.Persistence.CosmosDB
 
-> PS> Install-Package Orleans.Reminders.CosmosDB -prerelease
+> PS> Install-Package Orleans.Reminders.CosmosDB
 
-> PS> Install-Package Orleans.Streaming.CosmosDB -prerelease
+> PS> Install-Package Orleans.Streaming.CosmosDB
 
 .Net CLI:
 
-> \# dotnet add package Orleans.Clustering.CosmosDB -prerelease
+> \# dotnet add package Orleans.Clustering.CosmosDB
 
-> \# dotnet add package Orleans.Persistence.CosmosDB -prerelease
+> \# dotnet add package Orleans.Persistence.CosmosDB
 
-> \# dotnet add package Orleans.Reminders.CosmosDB -prerelease
+> \# dotnet add package Orleans.Reminders.CosmosDB
 
-> \# dotnet add package Orleans.Streaming.CosmosDB -prerelease
+> \# dotnet add package Orleans.Streaming.CosmosDB
 
 Paket: 
 
-> \# paket add Orleans.Clustering.CosmosDB -prerelease
+> \# paket add Orleans.Clustering.CosmosDB
 
-> \# paket add Orleans.Persistence.CosmosDB -prerelease
+> \# paket add Orleans.Persistence.CosmosDB
 
-> \# paket add Orleans.Reminders.CosmosDB -prerelease
+> \# paket add Orleans.Reminders.CosmosDB
 
-> \# paket add Orleans.Streaming.CosmosDB -prerelease
+> \# paket add Orleans.Streaming.CosmosDB
 
 # Configuration
 
@@ -130,6 +131,26 @@ For further details on partitioning in CosmosDB see https://docs.microsoft.com/e
 ### Backwards compatibility
 The change will not affect existing systems. Configuring a custom `IPartitionKeyProvider` for an existing system will throw a BadGrainStorageConfigException stating incompatibility. To start using a custom partition key provider the old documents must be updated with a `/PartitionKey` property where the value is set using the same functionality as in the `GetPartitionKey` implementation in the custom `IPartitionKeyProvider`. Once all documents are updated, the data must be migrated to a new CosmosDB collection using Azure Data Factory, CosmosDB Migration tool or custom code with colletions PartitionKey set to `PartitionKey`
 
+### Migration from 1.3.0 to 3.0.0
+
+With the 3.0.0 release, two breaking changes requires manual changes if you have pre-existent data:
+
+#### Remove stored procedures
+
+Reminders, Streaming and Persistence providers doesn't rely on Stored Procedures anymore. You can safely remove them from your databases once you migrate to 3.0.0. The only stored procedures that are still used, are the ones from the Clustering packages since Orleans membership protocol requires some atomic operations that are only possible on CosmosDB by using stored procedures. All the rest can be killed.
+
+#### Reminders collection
+
+Before 3.0.0, the reminders provider used to use a non-partitioned collection. Recently Microsoft requires that everyone using CosmosDB to migrate to partitioned collections. If you have data on your old collection, you need to migrate this data to a new one.
+
+The new structure is defined as follow:
+
+- `id` field: `$"{grainRef.ToKeyString()}-{reminderName}"`
+- `PartitionKey` field: `$"{serviceId}_{grainRef.GetUniformHashCode():X8}"`
+- Other fields remain the same.
+
+This data migration can be performed whatever the way you prefer, as long as the `id` and `PartitionKey` fields are formated the way described here. The partition key path of the new collection must be `/PartitionKey`.
+
 ### Indexing
 The current indexing fork relies on CosmosDB stored procedures for lookup. As stored procedures must be executed against a specific partition, the use of custom partition key builders is not compatible with the Orleans indexing fork. 
 

diff --git a/StartEmulator.cmd b/StartEmulator.cmd
@@ -0,0 +1,6 @@
+@echo OFF
+set containerName=azure-cosmosdb-emulator
+set hostDirectory=%LOCALAPPDATA%\azure-cosmosdb-emulator.hostd
+md %hostDirectory% 2>nul
+REM docker run --name %containerName% --memory 2GB --mount "type=bind,source=%hostDirectory%,destination=C:\CosmosDB.Emulator\bind-mount" -P --interactive --tty microsoft/azure-cosmosdb-emulator
+docker run --name %containerName% --memory 2GB --mount "type=bind,source=%hostDirectory%,destination=C:\CosmosDB.Emulator\bind-mount" -p 8081:8081 -p 8900:8900 -p 8901:8901 -p 8979:8979 -p 10250:10250 -p 10251:10251 -p 10252:10252 -p 10253:10253 -p 10254:10254 -p 10255:10255 -p 10256:10256 -p 10350:10350 microsoft/azure-cosmosdb-emulator