Research computing is a growing need and AWS cloud enables researchers to process big data with scalable computing in a secure and flexible manner. While Cloud computing is a powerful platform it also brings complexity with new tools, nomenclature and multiple options that distract researchers. Relevance Lab is partnering with AWS Public sector group and some leading US universities to create a frictionless “Research Data Platform (RDP)” leveraging open-source solutions.

Service Workbench from AWS is a powerful open-source solution for enabling research in cloud. Customers around the globe are already using this solution for common use cases.

• Enable researchers to use AWS Cloud with Self-service capabilities and common catalog of tools like EC2, SageMaker, S3, Studies data etc.
• Use common Data Analysis tools like RStudio in a secure and scalable manner.
• Setup a “Trusted Research Environment” in cloud for research with additional controls that enforce Ingress/Egress data restrictions for compliance.

While Service Workbench provides a good foundation platform for research, it also had some challenges based on feedback from early adopters mainly related to following:

• Complex setup requiring deep cloud know-how.
• An Admin centric User Experience not very Researcher friendly.
• Scalability challenges while adopting large scale research setups.
• Hard to customize.
• No enterprise support models available to guide customer through a Plan-Build-Run lifecycle.

Relevance Lab has built a modern and researcher friendly User Experience solution called “Research Data Platform” in collaboration with AWS and its early adopters extending the open-source foundation.

Key Functionalities of Research Data Platform
The primary goal is to drive frictionless research in cloud with following key features:

• Built as an open-source solution and made available to institutions interested in collaborating on a common Data Science Platform for research.
• “Project Centric” model enabling collaboration of researchers with common data, tools, and research goals in a self-service manner.
• Modern architecture with support for containers enabling researchers to bring their own tools covering Web-based software, Desktop-based tools, and Terminal-based solutions seamlessly accessed from Researcher Data Platform.
• Enable researchers to launch applications and choose configurations without knowledge of Cloud Infrastructure details for both regular and GPU workloads.
• Integrate with Datasets for research that are project centric and with a browser based easy interface to upload/download data for research.
• Ability to run multiple research projects across different AWS accounts with secure and scalable setup and guardrails.

The key functions flows needed for a Researcher are explained in the figure below:

Here is link for a demo of the solution.

Solution Architecture of Research Data Platform
The building blocks for the solution leverage the Service Workbench functionality and creates a separate Researcher Data Platform (RDP) layer for providing a UI driven application to Researchers roles and Admin users. The figure below captures the building blocks for this solution.

The solution consists of the following components:

• Webserver that serves the UI for the platform. The UI provides the entire researcher user experience whereby users can log in with their credentials and access the projects made available to them. Within the projects, users can launch applications that have been configured for them by the administrator. Users can choose the required configuration of the instances based on configurations created by the administrator.
• Research Data Platform DB. This database stores some of the configuration information and the mapping information required to faciliate the use of the underlying “Service Workbench” open-source software.
• Research Data Platform CLI. This command line interface allows the administrator to set up and configure projects, users, datasets, launchers and configurations easily.
• Service Workbench. This open-source software from AWS is the underlying API-driven engine that orchestrates and manages all the AWS resources on behalf of the user.

Deployment Architecture of Research Data Platform
The solution is deployed in an enterprise model for each customer in their AWS accounts and recommends the following architecture based on AWS Well Architected Framework as explained in figure below.

The deployment of the Research Data Platform consists of the following:

• One “Main” AWS account where RDP is deployed along with the Service workbench from AWS.
• Within the main account, Service Workbench is deployed as a serverless solution driven by APIs. It stores data in a DynamoDB database and uses AWS Service Catalog to manage and orchestrate resources. It uses Amazon S3 to create buckets that hold data.
• Within the main account, the Research Data Platform is deployed as a web server that serves the UI, along with an API backend that communicates with the Service Workbench.
• One or more project accounts are onboarded and can be used to create projects and access datasets.

Sample Screens for Research Data Platform
The key functionality for the solution is explained in some sample screens below.

Home Page: This is the first page that the user visits. From this page the user can choose to login to the Research Data Platform.

Projects Page: The projects page displays a card view of all the projects that the logged-in user is assigned to. Projects are set up by the administrator.

Each application that is useful to a researcher is set up as a launcher. Each launcher appears on the project workbench page as a card and the researcher can instantiate a session by clicking on the launcher card.

Files tab: This screen allows the researcher to browse the files in the datasets that are assigned to the project. A default storage area called project storage is available in every project. The project storage can also be browsed from this screen.

Launch Dialog: The user can select a configuration that is suitable for their research.

Project Details: The user can connect to Active sessions from the Workbench tab.

Sessions: An instance of a launcher is called a session. A user can connect to a session via the browser to access the application they need for conducting their research and analysis.

How Can New Customers Get Started?

• Reach out to Relevance Lab (write to rlcatalyst@relevancelab.com) for a quick discussion and demonstration of the standard solution
• We will capture an assessment of standard features vs know gaps for adopting the solution
• Engage on a Plan-Build-Run model based on deployment, enablement and operational readiness to start using Research in AWS cloud with simple and secure best practices
• Customers with standard needs can get started with a new setup in 8-10 weeks
• Relevance Lab will also provide on-going support and managed services

Conclusion
The Research Data Platform offers a comprehensive and researcher-friendly solution. It empowers researchers to process big data, perform data analysis, and conduct research efficiently in a secure and scalable manner. By bridging the gap between researchers and the AWS cloud, the RDP fosters innovation and advances scientific discovery in diverse domains.

References
Managing compute environments for researchers with Service Workbench on AWS
Using AWS Cloud for Research
Five ways to use AWS for research (starting right now)

While there is rapid momentum for every enterprise in the world in consuming more Cloud Assets and Services, there is still lack of maturity in adopting an “Automation-First” approach to establish Self-Service models for Cloud consumptions due to fear of uncontrolled costs, security & governance risks and lack of standardized Service Catalogs of pre-approved Assets & Service Requests from Central IT groups. Lack of delegation and self-service has a direct impact on speed of innovation and productivity with higher operations costs.

Working closely with AWS Partnership we have now created a flexible platform for driving faster adoption of Self-Service Cloud Portals. The primary needs for such a Self-Service Cloud Portal are the following.

• Adherence to Enterprise IT Standards
  • Common architecture
  • Governance and Cost Management
  • Deployment and license management
  • Identity and access management
• Common Integration Architecture with existing platforms on ITSM and Cloud
  • Support for ServiceNow, Jira, Freshservice and Standard Cloud platforms like AWS
• Ability to add specific custom functionality in the context of Enterprise Business needs
  • The flexibility to add business specific functionality is key to unlocking the power of self-service models outside the standard interfaces already provided by ITSM and Cloud platforms

A common way of identifying the need for a Self-Service Cloud portal is based on following needs.

• Does your enterprise already have any Self-Service Portals?
• Do you have a large user base internally or with external users requiring access to Cloud resources?
• Does your internal IT have the bandwidth and expertise to manage current workloads without impacting end user response time expectations?
• Does your enterprise have a proper security governance model for Cloud management?
• Are there significant productivity gains by empowering end users with Self-Service models?

Working with AWS partnership and an our existing customer, we see a growing need for Self-Service Cloud Portals in 2023 predominantly centred around two models.

• Enterprises with existing ITSM investments and need to leverage that for extending to Cloud Management
• Enterprises extending needs outside enterprise users with custom Cloud Portals

The roadmap to Self-Service Cloud portals is specific to every enterprise needs and needs to leverage the existing adoption and maturity of Cloud and ITSM platforms as explained below. With Relevance Lab RLCatalyst products we help enterprises achieve the maturity in a cost effective and expedited manner.

Examples of Self-Service Cloud Portals

• Scientific Research community is interested in a Research Gateway Solution
• University IT looks for a University in a Box Self-Service Cloud
• Enterprises using ServiceNow want to extend the internal Self-Service Portals
• Enterprises are also developing Hybrid Cloud Orchestration Portals
• Enterprises looking at building AIOps Portal needs monitoring, automation and service management
• Enabling Virtual Training Labs with User and Workspace onboarding
• Building an integrated Command Centre requires an Intelligent Monitoring portal
• Enterprise Intelligent Automation Portal with ServiceNow Connector

Explained below is a sample AWS Self-Service Cloud for driving Scientific Research.

Getting started
To make is easier for enterprises for experiencing the power of Self-Service Cloud Portals we are offering two options based on enterprise needs.

• Hosted SAAS offering of using our Multi-tenant Cloud Portal with ability to connect to your existing Cloud Accounts and Service Catalogs
• Self-Hosted RLCatalyst Cloud Portal product with option to engage us for professional services on customizations, training, initial setup & onboarding needs

Pricing for the SAAS offering is based on user based monthly subscription while for self-hosting model an enterprise support model pricing is available for the open source solution that allows enterprises the flexibility to use this solution without proprietary lock-ins.

The typical steps to get started are very simple covering the following.

• Setup an organization and business units or projects aligned with your Cloud Accounts for easy billing and access control tracking
• Setup users and roles
• Setup Budgets and controls
• Setup standard catalog of items for users to order
• With the above enterprises are up to speed to use Self-Service Cloud Portals in less than 1-Day with inbuilt controls for tracking and compliance

Summary
Cloud Portals for Self-Service is a growing need in 2023 and we see the momentum continuing for next year as well. Different market segments have different needs for Self-Service Cloud portals as explained in this Blog.

• Scientific Research community is interested in a Research Gateway Solution
• University IT looks for a University in a Box Self-Service Cloud
• Enterprises using ServiceNow want to extend the internal Self-Service Portals
• Enterprises are also developing Hybrid Cloud Orchestration Portals
• Enterprises looking at building AIOps Portal needs monitoring, automation and service management
• Enabling Virtual Training Labs with User and Workspace onboarding
• Building an integrated Command Centre requires an Intelligent Monitoring portal
• Enterprise Intelligent Automation Portal with ServiceNow Connector

We provide pre-build solutions for Self-Service Cloud Portals and a base platform that can be easily extended to add new functionality for customization and integration. A number of large enterprises and universities are leveraging our Self-Service Cloud portal solutions using both existing ITSM tools (Servicenow, Jira, Freshservice) and RLCatalyst products.

To learn more about using AWS Cloud or ITSM solutions for Self-Service Cloud portals contact marketing@relevancelab.com

Provide researchers access to secure RStudio instances in the AWS cloud by using Amazon issued certificates in AWS Certificate Manager (ACM) and an Application Load Balancer (ALB)

Cloud computing offers the research community access to vast amounts of computational power, storage, specialized data tools, and public data sets, collectively referred to as Research IT, with the added benefit of paying only for what is used. However, researchers may not be experts in using the AWS Console to provision these services in the right way. This is where software solutions like Service Workbench on AWS (SWB) make it possible to deliver scientific research computing resources in a secure and easily accessible manner.

RStudio is a popular software used by the Scientific Research Community and supported by Service Workbench. Researchers use RStudio very commonly in their day-to-day efforts. While RStudio is a popular product, the process of installing RStudio securely on AWS Cloud and using it in a cost-effective manner is a non-trivial task, especially for Researchers. With SWB, the goal is to make this process very simple, secure, and cost-effective for Researchers so that they can focus on “Science” and not “Servers” thereby increasing their productivity.

Relevance Lab (RL), in partnership with AWS, set out to make the experience of using RStudio with Service Workbench on AWS simple and secure.

Technical Solution Goals

1. A researcher should be able to launch an RStudio instance in the AWS cloud from within the Service Workbench portal.
2. The RStudio instance comes fully loaded with the latest version of RStudio and a variety of other software packages that help in scientific research computing.
3. The user launches a URL to the RStudio from within the Service Workbench. This URL is a unique URL generated by SWB and is encoded with an authentication token that ensures that the researcher can access the RStudio instance without remembering any passwords. The URL is served over SSL so that all communications can be encrypted in transit.
4. Maintaining the certificates used for SSL communication should be cost-effective and should not require excessive administrative efforts.
5. The solution should provide isolation of researcher-specific instances using allowed IP lists controlled by the end-user.

Comparison of Old and New Design Principles to make Researcher Experience Frictionless
The following section summarizes the old design and the new architecture to make the entire researcher experience frictionless. Based on feedback from researchers, it was felt that the older design required a lot of setup complexity and lifecycle upgrades for security certificate management, slowing down researchers productivity. The new solution makes the lifecycle simple and frictionless along with smart and innovative features to keep ongoing costs optimized.

The diagram below explains the interplay between different design components.

Secure and Scalable Solution Architecture
Keeping in mind the above design goals, a secure and scalable architecture is implemented that solves the problem of shared groups using products like RStudio requiring secure HTTPS access without the overheads of individual certificate management. The architecture also enables sharing the same concept for all future researcher products with similar needs without any additional implementation overheads resulting in increased productivity and lower costs.

The Relevance Lab team designed a solution centered on an EC2 Linux instance with RStudio and relevant packages pre-installed and delivered as an AMI.

1. When the instance is provisioned, it is brought up without a public IP address.
2. All traffic to this instance is delivered via an Application Load Balancer (ALB). The ALB is shared across multiple RStudio instances within the same account to spread the cost over a larger number of users.
3. The ALB serves over an SSL link secured with an Amazon-issued certificate which is maintained by AWS Certificate Manager.
4. The ALB costs are further brought down by provisioning it on demand when the first RStudio instance is provisioned. Conversely, the ALB is de-provisioned when the last RStudio instance is de-provisioned.
5. Traffic between the ALB and the RStudio instance is also secured with an SSL certificate which is self-signed but unique to each instance.
6. The ALB listener rules enforce the IP allowed list configured by the user.

Conclusion
Both SWB and Relevance Lab RLCatalyst Research Gateway teams are committed to making scientific research frictionless for researchers. With a shared goal, this new initiative speeds up collaboration and will help provide new innovative open-source solutions leveraging Service Workbench on AWS and partner-provided solutions like this RStudio with ALB from Relevance Lab. The collaboration efforts will soon be adding more solutions covering Genomic Pipeline Orchestration with Nextflow, use of HPC Parallel Cluster, and secure research workspaces with AppStream 2.0, so stay tuned.

To get started with RStudio on SWB provided by Relevance Lab use the following link:
Relevance Lab Github Repository for SWB Templates

For more information, feel free to contact marketing@relevancelab.com.

References
Service Workbench on AWS for driving Scientific Research
Service Workbench on AWS Documentation
Service Workbench on AWS Github Repository
RStudio Secure Architecture Patterns
Relevance Lab Research Gateway