Thursday, January 4, 2018

DotNet Core script case sensitivity on Linux systems, like AWS AMI

I recently ran into an issue where I was unable to deploy a DotNet Core 1.0 application in AWS using their CodeStar / CodePipeline / CodeDeploy / CodeBuild tooling.  The REST service started off as a simple demo .Net 1.x ASP.Net that was generated as a demo in AWS CodeStar.  I converted the source code, build and deployment components to DotNet 2.0. It worked great.

Problem

The service build / deployment broke in AWS when I added Swagger endpoint documentation that included information from C# XML documentation generated and stored an XML file. Startup failed saying they couldn't find the <ServiceName>.xml documenation file. This manifested as a build failure in CodePiepline/CodeDeploy.

  • bin\Release\netcoreapp2.0\TokenService.xml not found
    

Builds ran and deployed fine on my local machine in Debug and Release modes.

Cause

Microsoft's recommended "dotnet publish" command line may generate conflicting path with csproj files when using manipulating and consuming files sitting on a file system that is case sensitive.  File locations inside scripts and properties may contain conflicting capitalization. This is no problem on a case-insensitive files systems like NTFS.  It can be a problem on Linux systems..

I found this when running DotNet Core 2.x applications on an AWS on Ubuntu while enabling Swagger.  C# applications can generate XML documentation from the method and class API comments. Swagger can consume this XML in order to augment the default swagger comment behavior. That XML file must be in the right location with the right cased path.

XML Documentation file is generated at compile time and packaged with the service.  That file is only visible if all the components have the same path/case expectation.
  "/object_root/Release/netcore20/foo.xml" != "/object_root/release/netcore20/foo.xml"
.Net Microsoft standardized configurations to include "Debug" and "Production", not "debug" and "production". From the csproj file

  <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|AnyCPU'">
    <DocumentationFile>bin\Release\netcoreapp2.0\TokenService.xml</DocumentationFile>
  </PropertyGroup>

The following build command comes from AWS CodeStar templates and several wikipages. Note: the _lower case_ 'r' of the -o option.

  • dotnet publish -c release -o <output> <csproj>
  • creates
  • release\netcoreapp2.0\foo.xml
The configuration should be Release with a _capital R_.
  • dotnet publish -c Release -o ./build_output TokenService/TokenService.csproj
  • creates
  • Release\netcoreapp2.0\foo.xml

Sample  buildspec.yml for DotNet Core 2.0 with sSagger

Here is the updated buildspec file.  Note the dotnet publish output parameter

version: 0.2
phases:  pre_build:    commands:      - echo Using tooling dotnet version `dotnet --version`
      - echo Restore started on `date`
      - dotnet restore TokenService/TokenService.csproj
  build:    commands:      - echo Build started on `date`
      - dotnet publish -c Release -o ./build_output TokenService/TokenService.csproj
artifacts:  files:    - TokenService/build_output/**/*
    - scripts/**/*
    - appspec.yml

Code

You can find a .Net 2.0 compatible project that deploys on AWS Linux on this FreemanSoft GitNHub  repository.  
  • See the buildspec.yml for the right dotnet publish syntax. 
  • See the install_dotnetcore script to see how to instal .Net Core 2.0 on a Linux AMI

Create 2018 Jan 04

Wednesday, November 29, 2017

Browsers implement CORS to protect users, not servers

CORS is designed to protect users from cross site attacks where one site has the browser execute code to connect to another site without the user recognizing that it happened. CORS relies on the Web Browser to recognize and block disallowed cross site requests.  Server side CORS does not block service requests or protect a web site from direct interrogation and or programmatic attack.

Applications and Internet facing services cannot rely on CORS for general site protection. They can rely on CORS to protect the site when the site is connected to by a web browser.    Companies should not consider CORS any type of secure Authorization model. They should implement CORS policies that provide least privilege where possible.

Specified by Server, Implemented by Browser

Application servers can say that they allow connections from any browser no matter where that original browser session / page originally resided. This makes it simple for any page to aggregate information from multiple back-end services on multiple sites.

Application servers can say that they allow connections from a browser only when an original browser session / page were started on an approved web site. The basic flow looks like

  1. A user connects to a web site.
  2. The browser downloads HTML and Javascript.
  3. The browser makes a call for data from the original domain. The request is allowed because the page and data are from the same origin. 
  1. The browser requests data from a 2nd site.
  2. The browser sends a CORS request to the second site asking for a list of domains that are allowed to call the services on that site.
  3. The 2nd site CORS response describes which domains are allowed to connect for this cross-origin scenario.
  4. If not allowed, the browser aborts the call to services outside the page's origin domain with a return code of "Not Authorized"


NOTE: The browser implements cross-site authorization protection. The server specifies the desired protection level and the browser tries to implement it.

Specified by Server, Ignored by Appplications

Remember that CORS is implemented in the Browser to protect the User. CORS restrictions are not honored by web client libraries or the applications that sit on them.

CORS Specification: Allowing Cross Origin Requests

There really only three levels of cross origin that are allowed. They are specific and different: 

  • All : A server can say that it allows requests from pages and applications that originate in all other domains.  Examples uses include: Shared services, advertising services and other public endpoints.
  • None: A server can say that "no cross origin requests are allowed". This tells the browser that the service is only available to content/pages that originate on the same web server (FPDN).
  • Specified Exactly: A server can enable cross-site requests from specific hosts. Companies may use this setting to white-list their own cooperative applications while disabling request from outside organizations. Note that each host has to be listed here.
Wild cards are not supported.  This means that "*.domain.com" does not work.


Scenario: Multiple Enterprise Applications

A company could have a set of applications that exist as static content on web servers. The same company could put out a set of services to be shared across the organization.

Every web service host would list all internal Static Content host names in their CORS definitions.

This would let all applications share services while blocking consumption by non-firm web applications.










Working around CORS

CORS protection is all about "Cross Origin". It has no impact if everything looks like it comes from the same origin.  An alternative of building cooperative applications without explicitly listing hosts is to proxy all static content and and  HTTP/HTTPS web services through a single proxy. This would make all requests look like they came from the same domain.







    This post on YouTube



    References

    • https://en.wikipedia.org/wiki/Cross-origin_resource_sharing
    • https://www.owasp.org/index.php/CORS_OriginHeaderScrutiny
    Created 2017/11