New – AWS Data Exchange for Amazon Redshift

Again in 2019 I instructed you about AWS Information Trade and confirmed you learn how to Discover, Subscribe To, and Use Information Merchandise. At this time, you possibly can select from over 3600 knowledge merchandise in ten classes:

In my introductory publish I confirmed you ways may subscribe to knowledge merchandise after which obtain the info units into an Amazon Easy Storage Service (Amazon S3) bucket. I then recommended varied choices for additional processing, together with AWS Lambda capabilities, a AWS Glue crawler, or an Amazon Athena question.

At this time we’re making it even simpler so that you can discover, subscribe to, and use third-party knowledge with the introduction of AWS Information Trade for Amazon Redshift. As a subscriber, you possibly can immediately use knowledge from suppliers with none additional processing, and no want for an Extract Rework Load (ETL) course of. Since you don’t need to do any processing, the info is all the time present and can be utilized immediately in your Amazon Redshift queries. AWS Information Trade for Amazon Redshift takes care of managing all entitlements and funds for you, with all costs billed to your AWS account.

As a supplier, you now have a brand new strategy to license your knowledge and make it accessible to your clients.

As I used to be penning this publish, it was cool to understand simply what number of current facets of Redshift, and Information Trade performed central roles. As a result of Redshift has a clear separation of storage and compute, together with built-in knowledge sharing options, the info supplier allocates and pays for storage, and the info subscriber does the identical for compute. The supplier doesn’t must scale their cluster in proportion to the dimensions of their subscriber base, and may give attention to buying and offering knowledge.

Let’s check out this function from two vantage factors: subscribing to an information product, and publishing a knowledge product.

AWS Information Trade for Amazon Redshift – Subscribing to a Information Product
As a knowledge subscriber I can flick thru the AWS Information Trade catalog and discover knowledge merchandise which are related to my enterprise, and subscribe to them.

Information suppliers may create personal gives and lengthen them to me for entry by way of the AWS Information Trade Console. I click on My product gives, and assessment the gives which were prolonged to me. I click on on Proceed to subscribe to proceed:

Then I full my subscription by reviewing the supply and the subscription phrases, noting the info units that I’ll get, and clicking Subscribe:

As soon as the subscription is accomplished, I’m notified and may transfer ahead:

From the Redshift Console, I click on Datashares, choose Subscriptions, and I can see the subscribed knowledge set:

Subsequent, I affiliate it with a number of of my Redshift clusters by making a database that factors to the subscribed datashare, and use the tables, views, and saved procedures to energy my Redshift queries and my functions.

AWS Information Trade for Amazon Redshift – Publishing a Information Product
As a knowledge supplier I can embody Redshift tables, views, schemas and user-defined capabilities in my AWS Information Trade product. To maintain issues easy, I’ll create a product that features only one Redshift desk.

I exploit the spiffy new Redshift Question Editor V2 to create a desk that maps US space codes to a metropolis and a state:

Then I look at the record of current datashares for my Redshift cluster, and click on Create datashare to make a brand new one:

Subsequent, I’m going by means of the standard course of for making a datashare. I choose AWS Information Trade datashare, assign a reputation (area_code_reference), decide the database throughout the cluster, and make the datashare accessible to publicly accessible clusters:

Then I scroll down and click on Add to maneuver ahead:

I select my schema (public), choose to incorporate solely tables and views in my datashare, after which add the area_codes desk:

At this level I can click on Add to wrap up, or Add and repeat to make a extra advanced product that comprises further objects.

I affirm that the datashare comprises the desk, and click on Create datashare to maneuver ahead:

Now I’m prepared to begin publishing my knowledge! I go to the AWS Information Trade Console, increase the navigation on the left, and click on Owned knowledge units:

I assessment the Information set creation steps, and click on Create knowledge set to proceed:

I choose Amazon Redshift datashare, give my knowledge set a reputation (United States Space Codes), enter an outline, and click on Create knowledge set to proceed:

I create a revision referred to as v1:

I choose my datashare and click on Add datashare(s):

Then I finalize the revision:

I confirmed you learn how to create a datashare and a dataset, and to publish a product utilizing the console. In case you are publishing a number of merchandise and/or making common revisions, you possibly can automate all of those steps utilizing the AWS Command Line Interface (CLI) and the Amazon Information Trade APIs.

Preliminary Information Merchandise
A number of knowledge suppliers are working to make their knowledge merchandise accessible to you thru AWS Information Trade for Amazon Redshift. Listed here are among the preliminary choices and the official descriptions:

• FactSet Provide Chain Relationships – FactSet Revere Provide Chain Relationships knowledge is constructed to reveal enterprise relationship interconnections amongst corporations globally. This feed supplies entry to the advanced networks of corporations’ key clients, suppliers, opponents, and strategic companions, collected from annual filings, investor shows, and press releases.
• Foursquare Locations 2021: New York Metropolis Pattern – This trial dataset comprises Foursquare’ss built-in Locations (POI) database for New York Metropolis, accessible as a Redshift Information Share. Immediately load Foursquare’s Locations knowledge in to a Redshift desk for additional processing and evaluation. Foursquare knowledge is privacy-compliant, uniquely sourced, and trusted by prime enterprises like Uber, Samsung, and Apple.
• Mathematica Medicare Pilot Dataset – Mixture Medicare HCC counts and prevalence by state, county, payer, and filtered to the diabetic inhabitants from 2017 to 2019.
• COVID-19 Vaccination in Canada – This itemizing comprises pattern datasets for COVID-19 Vaccination in Canada knowledge.
• Revelio Labs Workforce Composition and Traits Information (Trial knowledge) – Perceive the workforce composition and developments of any firm.
• Facteus – US Card Shopper Cost – CPG Backtest – Historic pattern from panel of SKU-level transaction element from money and card transactions throughout lots of of Shopper-Packaged Items bought at over 9,000 city comfort shops and bodegas throughout the U.S.
• Decadata Argo Provide Chain Trial Information – Provide chain knowledge for CPG corporations delivering merchandise to US Grocery Retailers.

— Jeff;