Amazon’s Elastic Compute Cloud (EC2) uses auction-based spot pricing to sell spare capacity, allowing users to bid for cloud resources at a highly reduced rate. Amazon sets the spot price dynamically and accepts user bids above this price. Jobs with lower bids (including those already running) are interrupted and must wait for a lower spot price before resuming.

Spot pricing opens up an auction-based market in which cloud providers can dynamically provision data center resources to meet user demand and users can develop bidding strategies that lower their cloud resource costs. This cloud bidding project focuses on the mechanism of the cloud provider setting spot prices and user bidding strategies. Bidding higher prices can reduce the probability of interruptions, but cannot entirely prevent potential interruptions. We approach our solutions in three steps: (1) modeling the cloud provider’s setting of the spot price and matching the model to historically offered prices, (2) deriving optimal bidding strategies for different job requirements and interruption overheads, and (3) adapting these strategies to MapReduce jobs with master and slave nodes having different interruption overheads. We find that spot pricing reduces user cost by 90% with a modest increase in completion time compared to on-demand pricing.

Publications

L. Zheng, C. Joe-Wong, C. W. Tan, M. Chiang and X. Wang, How to Bid the Cloud, ACM SIGCOMM, 2015

Edge Artificial Intelligence and Machine Learning

• Secure Machine learning
• Edge AI Test Bed
• Hurts to Be Too Early: Benefits and Drawbacks of Communication in Multi-Agent Learning.
• Robotic Intervention Learning at the Edge

Fog Computing

• Clouds with Deadlines
• Completion-time-aware Scheduler
• Securing Tor
• Multi-resource fairness
• LTE Multicast
• Massive Machine-Type Communications
• Coded Computing
• Client Control of HetNets
• Shred and Spread
• Big Data Sharing
• Networked Drone Cameras

Smart Data Pricing

• Time-dependent Pricing
• Cloud Bidding
• Sponsored Data
• Cloud Virtual Service Provider
• Traded Data Plans
• Quota-aware Video Adaptation

Social Learning Networks

• Personalized Thread Recommendations for MOOC Forums
• Individualization
• Engagement Modeling
• Clickstream Analytics
• Social Learning Efficiency