How to handle ransomware attack on your AWS EC2 Instance

Handling a ransomware attack on your AWS account through commands can be complex and require expertise in managing AWS infrastructure. However, here are some steps you can follow using AWS CLI (Command Line Interface) to mitigate the damage:

1. Stop and isolate infected instances: Use the AWS CLI command to stop the infected instances immediately to prevent the spread of the ransomware:

python
aws ec2 stop-instances --instance-ids <instance-id>

Next, isolate the infected instances by changing their security group or subnet using the following command:

python
aws ec2 modify-instance-attribute --instance-id <instance-id> --groups <new-security-group-id>

2. Restore from backup: If you have backups, restore the affected data and systems from the most recent backup. You can use the AWS CLI command to create a new instance from the latest snapshot:

python
aws ec2 run-instances --image-id <snapshot-id> --instance-type <instance-type> --security-group-ids <security-group-id> --subnet-id <subnet-id>

3. Identify the source of the attack: Use AWS CloudTrail to identify the source of the attack by checking the logs of actions taken on your AWS account. You can use the AWS CLI command to search for CloudTrail events:

csharp
aws cloudtrail lookup-events --lookup-attributes AttributeKey=EventName,AttributeValue=<event-name>

4. Contact AWS support: Contact AWS support for assistance in cleaning up the infected instances and restoring access to your account if it has been locked by the attackers. You can use the AWS CLI command to open a support case:

css
aws support create-case --subject "<case-subject>" --service-code <service-code> --severity-code <severity-code> --category-code <category-code> --communication-body "<communication-body>"

5. Prevent future attacks: After recovering from the attack, take steps to prevent future attacks, such as implementing security best practices, regularly backing up your data, and using security tools such as firewalls and intrusion detection systems.

Overall, handling a ransomware attack on your AWS account through commands requires technical knowledge and expertise. It is recommended to seek assistance from AWS support or a professional AWS consultant.

How to Migrate MySQL Database to AWS RDS Aurora using DMS

 If you're looking to migrate your MySQL database to Amazon Aurora RDS, AWS provides a range of tools and services to make the process easier and more reliable. In this blog post, we will discuss how to use AWS Schema Conversion Tool (SCT) and AWS Database Migration Service (DMS) to migrate your MySQL database to Aurora RDS.

Step 1: Assess the compatibility of your MySQL database

Before you begin the migration process, you need to assess the compatibility of your MySQL database with Aurora RDS. AWS SCT can help you with this task by analyzing your MySQL schema and generating a report that identifies any incompatibilities between MySQL and Aurora RDS. This step is essential as it helps you identify any issues before you begin the migration process.

Step 2: Convert your MySQL schema using AWS SCT

After you have assessed the compatibility of your MySQL database, you can use AWS SCT to convert your schema to a format that is compatible with Aurora RDS. AWS SCT can automate this process for you, reducing the risk of human error and saving you time.

Step 3: Use DMS to migrate your data

Once you have converted your schema, you can use AWS DMS to migrate your data to Aurora RDS. AWS DMS provides a range of options for migrating your data, including full load, incremental load, and ongoing replication. You can choose the best option for your specific requirements and budget.

Step 4: Monitor the migration process

As you migrate your data to Aurora RDS, it's important to monitor the process to ensure that it's running smoothly. AWS provides a range of tools and services that can help you monitor your migration, including CloudWatch and the DMS console. By monitoring the process, you can identify any issues and address them before they become a problem.

Step 5: Test and validate the migration

After you have migrated your data to Aurora RDS, you need to test and validate the migration to ensure that everything is working correctly. AWS provides a range of tools and services that can help you with this task, including the Amazon RDS console, which allows you to view the status of your Aurora RDS instances and perform various administrative tasks.

Conclusion

Migrating your MySQL database to Aurora RDS can be a complex task, but by using AWS SCT and DMS, you can simplify the process and make it faster and more reliable. AWS SCT can help you convert your MySQL schema to a format that is compatible with Aurora RDS, while AWS DMS provides a range of options for migrating your data. By following these steps and monitoring the process, you can migrate your MySQL database to Aurora RDS with minimal downtime and disruptions to your business operations.

Using IAM ROLE/AWS CLI Credential for Codecommit Repository

What is CodeCommit: 

CodeCommit is a managed version control service that hosts private Git repositories in the AWS cloud. To use CodeCommit, you configure your Git client to communicate with CodeCommit repositories.

You can use a different types of credentials with IAM for authentication supported by CodeCommit.

* Git Credentials - Username/Password pair you can use for CodeCommit Repo over HTTPS

* SSH Keys - public/Private key pair to be used with CodeCommit Repo over SSH

* AWS Cli Access keys - Temporary Access keys with AWS Cli credential helper to CodeCommit Repo over HTTPS

Access Architecture

Install git-remote-codecommit and configure the AWS CLI 

 

1. Install git-remote-codecommit

Before you install git-remote-codecommit make sure you have the latest version of pip.

        pip --version

        Update pip to latest version :

        curl -O https://bootstrap.pypa.io/get-pip.py

        python3 get-pip.py --user

        Install git-remote-codecommit using pip

              pip install git-remote-codecommit

 

3. Install and configure AWS CLI

 

Install latest version or AWS CLI

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"

unzip awscliv2.zip

sudo ./aws/install

3. Create AWS Profile for Credentials.

 Use AWS CLI to configure Profile Named or default. If you have only one account default profile is okay, but working in an Enterprise environment means you have more than one AWS Account, and Managing all can be challenging, so it's recommended to have a meaningful name for your profile. 

             

            aws configure set role_arn arn:aws:iam::111111111111:role/AWS-Role-Name --profile AWS-Account-A
            aws configure set source_profile default --profile AWS-Account-A
            aws configure set role_session_name "YourName" --profile AWS-Account-A

          If you want to verify the Changes, manually view or edit file  ~/.aws/config, if will look similar to following.

                [default]

                region = us-east-1

                output = json

                [profile AWS-Account-A]

                source_profile = default

                role_session_name = YourName

                role_arn = arn:aws:iam::111111111111:role/AWS-Role-Name

Connect to AWS CodeCommit using AWS Profile/STS Token

  

            Test AWS Profile is working okay and is connecting successfully.

$ aws sts get-caller-identity --profile AWS-Account-A

$ git clone codecommit://AWS-Account-A@AWSCodeCommitRepoName

Building Containerized micro services in AWS with Multi Region High Availablity Architecture

we will look into designing and building AWS-based multi-region active-passive as well as Active-Active architecture and utilize microservices deployed using Docker containers in EKS / ECS as well as AWS Fargate offering.

Requirements and Assumptions

To facilitate the above, we will make up some sample requirements and assumptions, which will require a multi-region active-active deployment.

• Tier 1 Web Application: 99.99% availability. Annual Downtime: 52 mins.
• RPO (Recovery Point Objective): Near Real-time.
• RTO (Recovery Time Objective): n/a (Always active).
• Application must have Multi-Region deployments to serve global user traffic.
• User traffic must be served from their own, or nearest AWS region.
• Components of the app:
• Rich Internet Application (RIA) built using Angular 7+. Angular app will access micro services deployed in Docker containers.
• ElasticSearch will be used as a search engine, and will be exposed to the app via a micro service.
• Micro services will be developed using Java (Spring Boot), and deployed in Docker containers.
• There is no advisory on infrastructure sizing for any of the components (as it will depend on the non-functional requirements of a specific application), and hence ignored in this article.

What is an Active-Active Architecture?

• An active/active system is a network of independent processing nodes, each having access to a (..) replicated database such that all nodes can participate in a common application [4].
• For all practically purpose, we will have two or more ‘sets’ of applications running in two more geographically dispersed different AWS regions. Both regions will have all the required components and data, so that in case one region fails, another can automatically take over user traffic from the failed region without having a downtime for the globally dispersed app users.

Active-Active vs Active-Passive Architectures

• Active-active architecture requires that we have all components running in both regions at the same time
• Active-passive architecture suggests we have all components running only in the ‘active’ region/data centre, but have mechanisms in place to automatically (or semi-automatically/manually) failover to to passive region and install/start components/application in the passive region as quickly as possible.
• From an AWS perspective, this means that you have already configured VPC (security groups, etc) in the passive region, have some sort of data backups in that region, and can quickly deploy and start your components in that region (when failover occurs). There is a whole lot of other issues to consider (such as availability of data snapshots, etc) in the passive region, so that you can recover data and re-create database and other necessary app data in the passive region, and make it active. Those are beyond the scope of this article.

Rationale for Multi-Region Active-Active Architecture

Below are the some of the key reasons why you might consider building a multi-region active-active architecture:

• Improved latency for end users – Closer your backed is to end users, the better customer experience they will have.
• Disaster recovery – Mitigate against a whole region failure.
• Business Requirements – Tier 1 apps with 99.99%+ availability non-functional requirements.

Key AWS Services

Below are some of the key AWS services that can be leveraged to achieve some of the above requirements:

• Route 53 -DNS service, Traffic Flow management across AWS regions
• CloudFront – Global Content Delivery Network (CDN)
• AWS Aurora Global Database – Designed for globally distributed applications, allowing a single Amazon Aurora database to span multiple AWS regions. Cross Region: Recovery Point Objective (RPO) of 1 second and a Recovery Time Objective (RTO) of less than 1 minute (in case of a region failure)
• AWS ElasticSearch (ES) – Managed service to deploy, secure, and operate Elasticsearch at scale with zero down time
• AWS Elastic Container Service (ECS) – Container orchestration service that supports Docker containers and allows you to easily run and scale containerised applications on AWS. With ECS, you can either use EC2 instances or AWS Fargate to deploy your containers.
• AWS Elastic Container Registry (ECR) – Managed Docker container registry that makes it easy for developers to store, manage, and deploy Docker container images
• AWS Fargate – Compute engine for Amazon ECS that allows you to run containers without having to manage servers or clusters

Additionally a global solution will benefit from the following services (not shown in the deployment diagram):

• AWS Shield – Managed Distributed Denial of Service (DDoS) protection service
• AWS CloudTrail – Enables governance, compliance, operational auditing, and risk auditing of your AWS account
• Elastic Logstash Kibana (ELK) Stack – ELK stack to extract logs from different services in one central place for issue analysis
• AWS ElastiCache – Managed, Redis or Memcached-compatible in-memory data store. You can use it to offload traffic from your RDS Aurora DB or ElasticSearch cluster.

AWS Route 53 Routing Policies

AWS Route 53 provides several routing policies, which can be leveraged to route traffic to a certain region based on different factors. Some of the key policies relevant for a multi-region active-active architecture are listed below [1]:

• Failover routing policy – Use when you want to configure active-passive failover. For example, if US region fails, route US traffic to EU region (failover region for US).
• Geolocation routing policy – Use when you want to route traffic based on the location of your users.
• Geoproximity routing policy – Use when you want to route traffic based on the location of your resources and, optionally, shift traffic from resources in one location to resources in another.
• Latency routing policy – Use when you have resources in multiple AWS Regions and you want to route traffic to the region that provides the best latency.
• Multivalue answer routing policy – Use when you want Route 53 to respond to DNS queries with up to eight healthy records selected at random.

Multi-Region Active-Active Overall Architecture

• You can route a user’s API requests based on their geolocation (Route EU users to EU region)
• Define failover regions for each region (If EU region fails, route traffic to US region so that users can still access the app)

Detailed Deployment Architecture for a Region

Each AWS region will have a deployment architecture similar to the below.

• Bastion Host – There will be a bastion host in each region. Developers/admin can SSH to this host, and from there they can access RDS, Microservices and ElasticSearch. There is a Network Load Balancer (NLB) in front of the Bastion host, as you can have multiple low spec Bastion Hosts in two Availability Zones (AZ). If a bastion host instance is replaced by AWS, if would not affect your developers. They can still SSH using the same NLB DNS address.
• CloudFront – Global users will access the Angular app via CloudFront CDN for low latency downloads.
• AWS S3 – Angular based app will be deployment in S3.
• Router 53 – All user traffic will go via Route 53, which will make routing decisions regarding which region the users requests should be served from.
• AWS ALB (Application Load Balancers) – Each region will have its own ALB, which will route traffic to different micro services running in the Docker containers. For example, all product related web service calls will go to the ‘product’ micro services container. To achieve this, you will define a Target Group for each micro service, and map different request paths to a target group. For example, ‘/product/’ path could be mapped to a Target Group that contains all your containers running the Product micro service.
• ECS Fargate – It will contain all your containers running different micro services.
• AWS ES – It will contain for example product related indexed data, and will be used by the ‘search’ micro service.
• RDS Aurora DB – It will contain all your product and user data. Each region will have its own primary and standby replica. If primary goes down in a region, AWS will promote the standby as primary. If the whole region containing primary goes down, you will have to promote another regions RDS Aurora cluster as the primary for the Aurora Global DB.

Security Architecture

In each Availability Zone (AZ), there is 1 public subnet, and 2 private subnets to segregate components. Load balancers are in the public subnet, while rest (ECS containers, ElasticSearch, DB) are in private subnets.

Security Groups (SG)

Traffic flow between different components is controlled via Security Groups. A security group allows traffic only on a certain port to another security group.

• Application Load Balancer (ALB) SG – Allow traffic on port 443 from public internet
• Network ALB SG – allow traffic on port 22 from your office IP range.
• Bastion SG – allow traffic on port 22 only from Network ALB SG
• Elastic Search SG – allow traffic on port 9300 from Bastion SG and Application Load Balancer (ALB) SG
• ECS Fargate SG – allow traffic on port 8080 (running Spring Bootstrap) from ALB SG.
• Database SG – allow traffic on port 3306 from bastion SG and ECS Fargate SG.

Access to ElasticSearch on local machine

To access AWS ElasticSearch Service (ES) from a local machine, developers can create an SSH tunnel via the bastion host. That way, they can access Kibana and ElasticSearch REST API locally in their browser to diagnose any problems with the data contained in the search cluster.

For example. add below to your ~/.ssh/config file to access Kibana and ElasticSearch REST API locally on your machine.

Note: Please remember to replace the keypair file name, bastion host ALB public DNS name and private DNS name of your ElasticSearch cluster.

# Elasticsearch Tunnel
# https://www.jeremydaly.com/access-aws-vpc-based-elasticsearch-cluster-locally/
Host estunnel
HostName multi-bastion-lb-abc.elb.eu-west-2.amazonaws.com
User ec2-user
IdentitiesOnly yes
IdentityFile ~/.ssh/multi-keypair.pem
LocalForward 9200 vpc-multi-es-domain-abc.eu-west-2.es.amazonaws.com:443

After this you can access Kibana locally in the browser:

https://localhost:9200/_plugin/kibana/app/kibana#/management/kibana/index?_g=()

And test access to ES API via curl or use Postman to test different API requests.

$ curl -k https://localhost:9200/

Same kind of access could be configured to access RDS Aurora locally (for Dev environments).

AWS Route 53 Routing Policy

AWS provides traffic flow visual editor that can be used to to configure several routing policies as mentioned above in this article.

• For example, if you are using three regions US, EU, & APAC.
• You may direct all US users HTTP requests to your US region VPC using Geolocation based routing policy.
• To have automatic failover for different regions, you can for example define EU as a failover regions for US. This way, if the whole US region goes down, Route 53 will route traffic destined for US region to your failover EU regions VPC.
• You can repeat the same for EU and APAC regions.

Example Routing Policy

Multi-Region Active-Active Architecture Costs

• Running a redundant set of components across multiple regions is by no means a cheaper option. It is costly!
• Thats why majority of the applications are either deployed in a single region within Multi AZs or use an active-passive architecture where active region has all the required components, while passive region has just bare bone VPC and some data backups.
• So, unless you need to absolutely meet the active-active multi-region requirements as outlined in the beginning of this article, you may be fine just deploying in a single region.

Summary

• Multi-region active-active architectures are expensive to build and maintain, but are essential if your app requires low latency for your global users, is a Tier 1 app with 99.99% availability requirements, and requires built in disaster recovery capability across regions.
• AWS RDS Global DB simplifies the cross region DB replication, and facilitates active-active architecture.
• Using AWS Fargate along with ECS Registry (ECR) simplifies running containers in AWS, and deployment and updates of microservices containers. Developers can easily build & test a Docker container locally, and push it to ECR. From there, they can easily pushed to ECS without any downtime for existing microservices as ECS gradually replaced the old containers with the new containers.
• There are some tech choices made in this article, but please do your own due diligence to choose the correct technologies for your own specific use case. For example, using ECS vs Kubernetes, using EC2 or Fargate with ECS, using RDS Global Database vs DynamoDB Global Tables.

Application Containers

### Raw version of application container Demo.
### Will convert into full Detail blog by some unknown date.


sqlplus  "/ as sysdba"

show parameter create
show pdbs


CREATE  pluggable database  APPROOT AS APPLICATION CONTAINER ADMIN USER USER1 IDENTIFIED BY USER1;

 -- sqlplus  sys/welcome1@localhost:1522/APPROOT  as sysdba

alter session set container=APPROOT ;

alter pluggable database APPROOT open;

alter pluggable database application demo begin install '1.0';

create tablespace app datafile size 1m autoextend on next 1m;
create user app_user identified by app_user default tablespace app quota unlimited on app;

GRANT CREATE SESSION, CREATE TABLE TO  app_user;


create table app_user.tab1 SHARING=METADATA
(
col1 number,
col2 date,
col3 varchar2(20));

create table app_user.tab2 SHARING=DATA
(
col1 number,
col2 date,
col3 varchar2(20));


create table app_user.tab3 SHARING=EXTENDED DATA
(
col1 number,
col2 date,
col3 varchar2(20));

insert into app_user.tab1 select 1,sysdate,'meta-'||name from v$pdbs ;
insert into app_user.tab2 select 1,sysdate,'data-'||name from v$pdbs ;
insert into app_user.tab3 select 1,sysdate,'extend'||name from v$pdbs ;


commit;


colum app_name format a30

select app_name,app_version,app_status from DBA_APPLICATIONS ;

alter pluggable database application demo end install;





####

CREATE  pluggable database  APP1 ADMIN USER USER1 IDENTIFIED BY USER1;

alter pluggable database  APP1  open;


alter session set container=APP1;

select col1,col2,col3 from app_user.tab1;
select col1,col2,col3 from app_user.tab2;
select col1,col2,col3 from app_user.tab3;



 ALTER PLUGGABLE DATABASE APPLICATION ALL SYNC;

insert into app_user.tab1 select 2,sysdate,'meta-'||name from v$pdbs ;
insert into app_user.tab2 select 2,sysdate,'data-'||name from v$pdbs ;
insert into app_user.tab3 select 2,sysdate,'extend'||name from v$pdbs ;


commit;

create table app_user.app1tab4
(
col1 number,
col2 date,
col3 varchar2(20));





### another application pdbs

alter session set container=APPROOT ;

CREATE  pluggable database  APP2 ADMIN USER USER1 IDENTIFIED BY USER1;

alter pluggable database  APP2  open;


alter session set container=APP2;


 ALTER PLUGGABLE DATABASE APPLICATION ALL SYNC;


select col1,col2,col3 from app_user.tab1;
select col1,col2,col3 from app_user.tab2;
select col1,col2,col3 from app_user.tab3;


insert into app_user.tab1 select 2,sysdate,'meta-'||name from v$pdbs ;
insert into app_user.tab2 select 2,sysdate,'data-'||name from v$pdbs ;
insert into app_user.tab3 select 2,sysdate,'extend'||name from v$pdbs ;

commit;




select col1,col2,col3 from app_user.tab1;
select col1,col2,col3 from app_user.tab2;
select col1,col2,col3 from app_user.tab3;


create table app_user.app2tab5
(
col1 number,
col2 date,
col3 varchar2(20));







### root container
select app_name,app_version , app_status from DBA_APP_PDB_STATUS ;



==== clear demo
## connect to root Container

alter session set container=APPROOT ;

alter pluggable database APP1  close;
drop pluggable database APP1 including datafiles;

alter pluggable database APP2  close;
drop pluggable database APP2 including datafiles;


alter session set container=CDB$ROOT ;
alter pluggable database APPROOT  close;
drop pluggable database APPROOT including datafiles;

Oracle Database 12c Certified Master Upgrade Exam - OCM 12C

Recently I have attended the Oracle 12C OCM upgrade exam and i can tell you that it's not easy exam considering the amount of topics covered in one.

If you are also looking to attend in near future you need to be quick if planing to attend in Europe as all of Oracle education/Examination Centre are scheduled to close. last one will be Oracle Netherlands in Utrecht by end of April 2017.

Post April 2017 all exam are expected to conducted only in Reading, UK in Europe reason, not sure if UK will still be in EU that time. :-)

Even though 12.2 is out from some time on cloud and now in Premise as well from Feb 2017 but Exam is still tested against 12.1.0.2.0. But we might see some changes adapted to 12.2. We expect oracle will announce before adapting this.

During my preparation i noticed Oracle education website changed and in some cases i could not even find the topic details, however they seems to appear again now, so i thought to share with everyone before it disappear again there,

Passing score for this exam is :Overall 60% 

I remember not all module need 60% to pass, like Module 2  (Data and Performance Management) need only 32% but overall must be 60% , but this information is not available anymore on website. i will update if manage to find somewhere.

General Database and Network Administration, and Backup Strategy

• Create and manage pluggable databases
• Create and manage users, roles, and privileges
• Configure the network environment to allow connections to multiple databases
• Protect the database from loss of data due to any kind of failure
• Create and manage database configuration files

Data and Performance Management

• Modify materialized views
• Create a plugged-in tablespace by using the transportable tablespace feature
• Create partitioned tables
• Configure the database to retrieve all previous versions of the table rows
• Configure the Resource Manager
• Tune SQL statements
• Perform real application testing
• Create SQL Plan baselines

Data Guard

• Create a physical standby database
• Make the standby database available for testing
• Restore the standby database to its normal function
• Configure fast start failover

Grid Infrastructure and Real Application Clusters

• Install Oracle Grid Infrastructure
• Configure Oracle Flex ASM
• Create ASM disk groups
• Create and manage an ASM instance
• Create ACFS
• Start, stop, configure, and administer Oracle Grid Infrastructure
• Install the Oracle Database 12c software
• Create RAC databases
• Configure services

Happy Learning and Best of luck if you planning to attend in near future.

Please be aware OCM profiles are no longer hosted like with static URL, its now simply show your OTN profile. 

New URL is https://community.oracle.com/community/technology_network_community/certification/ocm where all OCM'  are displayed in very unstructured way. :-( , I Personally didn't like it, but doesn't really matter. 

--

Krishan Jaglan

OCM 12C 

Step by step Oracle Database 12.2 for Exadata - Download today

For those who are waiting for Oracle database 12cR2 ( Oracle 12.2), good news is that, it's available for download since 10 February 2017 from Oracle e-delivery website.

Here are the step by step instructions on downloading.


Step1: Go to https://edelivery.oracle.com and login using your Oracle Metalink username/Passsord.
Step2: Search for Oracle Database Enterprise Edition
Step3: Select Linux x86 - 64 from "Select platform" list

Step4 : Press continue
Step5:  click little down arrow key to see details and make sure your selection is correct.

Step6:  Press continue and accept oracle Term and Conditions.

and Oracle 12.2 is available to download and install on your Test system so you can start planning the upgrade when you fully tested against your application.


Please beware this download is available for Exadata  and it will be available for  standard x86-64 bit platform from 15-March 2017.

Here is the list of version available for different component.  you will notice all components are available for 12.2 except database client on windows platform which is still 12.1.0.2

Oracle Database Client (12.2.0.1.0 Exadata/SuperCluster)

V839967-01.zip Oracle Database 12c Release 2 Client (12.2.0.1.0) for Linux x86-64

V839968-01.zip Oracle Database 12c Release 2 Client (12.2.0.1.0) for Linux x86

Oracle Database Global Service Manager (12.2.0.1.0 Exadata/SuperCluster)

V840019-01.zip Oracle Database 12c Release 2 Global Service Manager (12.2.0.1.0) for Linux x86-64

Oracle Database Grid Infrastructure (12.2.0.1.0 Exadata/SuperCluster)

V840012-01.zip Oracle Database 12c Release 2 Grid Infrastructure (12.2.0.1.0) for Linux x86-64

Oracle Database Client (12.1.0.2.0)

V47121-01.zip Oracle Database 12c Release 1 Client (12.1.0.2.0) for Microsoft Windows x64 (64-bit)

V47124-01.zip Oracle Database 12c Release 1 Client (12.1.0.2.0) for Microsoft Windows (32-bit)

Oracle HTTP Server (12.2.1.1.0)

V266898-01.zip Oracle Fusion Middleware 12c (12.2.1.1.0) HTTP Server for Linux x86-64

Happy Upgrading. !!!!