You can translate the content of this page by selecting a language in the select box.
The AWS Certified Machine Learning Specialty validates expertise in building, training, tuning, and deploying machine learning (ML) models on AWS.
Use this App to learn about Machine Learning on AWS and prepare for the AWS Machine Learning Specialty Certification MLS-C01.
[appbox appstore 1611045854-iphone screenshots]
[appbox microsoftstore 9n8rl80hvm4t-mobile screenshots]
The App provides hundreds of quizzes and practice exam about:
– Machine Learning Operation on AWS
– Data Engineering
– Computer Vision,
– Exploratory Data Analysis,
– ML implementation & Operations
– Machine Learning Basics Questions and Answers
– Machine Learning Advanced Questions and Answers
– Countdown timer
– Machine Learning Cheat Sheets
– Machine Learning Interview Questions and Answers
– Machine Learning Latest News
The App covers Machine Learning Basics and Advanced topics including: NLP, Computer Vision, Python, linear regression, logistic regression, Sampling, dataset, statistical interaction, selection bias, non-Gaussian distribution, bias-variance trade-off, Normal Distribution, correlation and covariance, Point Estimates and Confidence Interval, A/B Testing, p-value, statistical power of sensitivity, over-fitting and under-fitting, regularization, Law of Large Numbers, Confounding Variables, Survivorship Bias, univariate, bivariate and multivariate, Resampling, ROC curve, TF/IDF vectorization, Cluster Sampling, etc.
Domain 1: Data Engineering
Create data repositories for machine learning.
Identify data sources (e.g., content and location, primary sources such as user data)
Determine storage mediums (e.g., DB, Data Lake, S3, EFS, EBS)
Identify and implement a data ingestion solution.
Data job styles/types (batch load, streaming)
Data ingestion pipelines (Batch-based ML workloads and streaming-based ML workloads), etc.
Domain 2: Exploratory Data Analysis
Sanitize and prepare data for modeling.
Perform feature engineering.
Analyze and visualize data for machine learning.
Domain 3: Modeling
Frame business problems as machine learning problems.
Select the appropriate model(s) for a given machine learning problem.
Train machine learning models.
Perform hyperparameter optimization.
Evaluate machine learning models.
Domain 4: Machine Learning Implementation and Operations
Build machine learning solutions for performance, availability, scalability, resiliency, and fault
Recommend and implement the appropriate machine learning services and features for a given
Apply basic AWS security practices to machine learning solutions.
Deploy and operationalize machine learning solutions.
Machine Learning Services covered:
AWS Deep Learning AMIs (DLAMI)
Amazon Fraud Detector
Other Services and topics covered are:
AWS ML application services
Language relevant to ML (for example, Python, Java, Scala, R, SQL)
Notebooks and integrated development environments (IDEs),
S3, SageMaker, Kinesis, Lake Formation, Athena, Kibana, Redshift, Textract, EMR, Glue, SageMaker, CSV, JSON, IMG, parquet or databases, Amazon Athena
Amazon EC2, Amazon Elastic Container Registry (Amazon ECR), Amazon Elastic Container Service, Amazon Elastic Kubernetes Service , Amazon Redshift
Important: To succeed with the real exam, do not memorize the answers in this app. It is very important that you understand why a question is right or wrong and the concepts behind it by carefully reading the reference documents in the answers.
Note and disclaimer: We are not affiliated with Microsoft or Azure or Google or Amazon. The questions are put together based on the certification study guide and materials available online. The questions in this app should help you pass the exam but it is not guaranteed. We are not responsible for any exam you did not pass.
- [R] Which NVIDIA A100 version was used for training the model?by /u/alf_Lafleur (Machine Learning) on December 9, 2023 at 10:58 am
In this publication, they mentioned the NVIDIA A100 for training, but I am not sure if the 40GB or the 80GB version. They added "memory" is 32GB, but maybe it is just the computer RAM. Any clues? And how would you match that with a less expensive GPU? https://github.com/sony/hFT-Transformer https://arxiv.org/pdf/2307.04305.pdf submitted by /u/alf_Lafleur [link] [comments]
- [R] Vision Grid Transformers for Document Layout Analysisby /u/GustaMusto (Machine Learning) on December 9, 2023 at 8:16 am
AlibabaResearch recently (Oct 2023) published a new model for Document Layout Analysis which sets a new benchmark in the task of Document Layout Analysis. Introduction - To fully leverage multi-modal information and exploit pre-training techniques to learn better representation for DLA, in this paper, we present VGT, a two-stream Vision Grid Transformer, in which Grid Transformer (GiT) is proposed and pre-trained for 2D token-level and segment-level semantic understanding https://arxiv.org/abs/2308.14978 Effect on LLM usage - VGT can dissect the page into different portions (headers, subheaders, titles, etc.) which can then be OCRed and passed to an LLM for RAG. With personal usage of VGT, it seems that even visually rich documents can be parsed easily with very little post-processing. submitted by /u/GustaMusto [link] [comments]
- [Discussion] Computational Learning Theory Problemby /u/StudentClassof2025 (Machine Learning) on December 9, 2023 at 6:50 am
Here is the problem I am interested in solving. I'm not sure where this problem is from but it looks like its from: A Gentle Introduction to Computational Learning Theory. Here's some of the work I've done to solve this problem but I'm still lost on how to approach it: https://preview.redd.it/i8thma5uu75c1.jpg?width=2048&format=pjpg&auto=webp&s=759e3d7b432dc0484f6566b9663451cea6828876 submitted by /u/StudentClassof2025 [link] [comments]
- [D] How do people know what the “best models” are right now?by /u/stuck-in-an-ide (Machine Learning) on December 9, 2023 at 6:43 am
For some reason everybody has been crazy hype about “MistralAI” releasing a new model. Is there a general website people go to, to compare which models are currently the “leading models”? Are there specific stats people look for, to figure out which models are the best? submitted by /u/stuck-in-an-ide [link] [comments]
- [D] AAAI Conference Decisions Out!by /u/benthehuman_ (Machine Learning) on December 9, 2023 at 5:23 am
No email notification for me, but it’s up in CMT 🙂 submitted by /u/benthehuman_ [link] [comments]
- Fine Tuning SDXL [P]by /u/Simple-Alps-7409 (Machine Learning) on December 9, 2023 at 2:30 am
Hi! New-ish to ML/AI here. I’m working on a project and am trying to figure out the best way to fine tune SDXL with a small set of face images, produce an output, and repeat with a new set of face images(many many times) I’ve looked into Replicate which has a really simple method of doing so and it seems feasible, but I’m unsure of how scalable or how cost effective that would be ( looks like each training would run me around $0.37). Any suggestions on where to look? Any help is greatly appreciated. submitted by /u/Simple-Alps-7409 [link] [comments]
- Group [D]iscussion on OpenAI's foundational CLIP Paper for Zero-Shot Image Classificationby /u/FallMindless3563 (Machine Learning) on December 8, 2023 at 11:23 pm
Hey all, we had a lively group discussion today on the 2021 CLIP paper from OpenAI. Every Friday we've been going over the fundamentals of a lot of the state of the art techniques used in Machine Learning today. Hoping to learn a little each week, and spot patterns we can apply to our own work. I feel like there's always a little nugget of information I didn't fully understand before reading the paper, so have been finding it helpful. Though it is not groundbreaking research as of this week, I think it's nice to take a step back and review the fundamentals as well as keeping up with the latest and greatest. Posted the notes and video recap are here if anyone finds it helpful: https://blog.oxen.ai/arxiv-dives-zero-shot-image-classification-with-clip/ Also would love to have anyone join us live on Fridays or suggest papers! We've got a pretty consistent and fun group of 400+ engineers and researchers popping in and out. submitted by /u/FallMindless3563 [link] [comments]
- [Research] Papers to read to stay up to dateby /u/MyFriendTheGuitar (Machine Learning) on December 8, 2023 at 10:53 pm
Hi there! I am familiar with how dnn, cnn, recurrent neural networks, gan, policy gradient reinforcement learning and transformer networks work. I am looking to expand my knowledge and stay up to date within the field. Do you have any recommendations on papers to read to stay up to date? I realise that there are vast amounts of papers being produced in this field, but I would appreciate if there are suggestions for «must not miss» papers. submitted by /u/MyFriendTheGuitar [link] [comments]
- [P] Labelling Interface for Audio Event Classification Training Databy /u/davegravy (Machine Learning) on December 8, 2023 at 10:39 pm
I'm interested in suggestions that will feed the design of an optimal interface for humans to label audio clips that will be used to form a training set used in audio event classification. For background, ultimately I want to identify what urban sound(s) are present in ~15 second .opus files. For certain classes (namely those associated with noisy construction-related equipment) it's much more important that inferences minimize false negatives. If it's relevant, the same audio recording device that's producing the clips I need to label will produce the clips I'll want to perform inferences on (i.e it's in the same physical location, same acoustic environment). Questions: I don't have a neural network selected yet, is it critical to select this before designing the labelling scheme or are best practices generalizable? Should I allow labelers to create their own labels if they aren't in my predefined set? Should I have an "unknown" label? Is it best to perform clustering first so that like-clips can be presented in sequence? Should I ask labelers to identify all classes present in each clip or just one? Is there value in being more specific with labels (e.g. "accelerating motorcycle engine" vs "idling motorcycle engine") than I need the end-inferences to be (e.g. "motorcycle")? Are there any other questions I should be asking? submitted by /u/davegravy [link] [comments]
- [R] Using Large Language Models for Hyperparameter Optimizationby /u/ImpossibleTeach880 (Machine Learning) on December 8, 2023 at 9:43 pm
We've spent a lot of effort tuning hyperparameters for large models, but what if they could also help us tune hyperparameters? Arxiv: https://arxiv.org/abs/2312.0452 submitted by /u/ImpossibleTeach880 [link] [comments]
- [R] QuIP#: SOTA 2 bit LLMsby /u/tsengalb99 (Machine Learning) on December 8, 2023 at 8:49 pm
We're pleased to introduce QuIP#, a new SOTA LLM quantization method that uses incoherence processing from QuIP (the paper) & lattices to achieve 2 bit LLMs with near-fp16 performance! Now you can run LLaMA 2 70B on a 24G GPU w/out offloading! QuIP# crushes all publicly available 2 bit PTQ methods on language modeling & zero shot tasks while being conceptually clean and simple. We’ve released quantized LLaMA, Mistral, and OpenHermes models, and a full codebase at https://github.com/Cornell-RelaxML/quip-sharp More information on how QuIP# works here https://cornell-relaxml.github.io/quip-sharp/. submitted by /u/tsengalb99 [link] [comments]
- [R] Paving the way to efficient architectures: StripedHyena-7B, open source models offering a glimpse into a world beyond Transformersby /u/hzj5790 (Machine Learning) on December 8, 2023 at 8:47 pm
https://www.together.ai/blog/stripedhyena-7b submitted by /u/hzj5790 [link] [comments]
- [D] Out of distribution is equivalent to data drift?by /u/Relevant-Fortune-810 (Machine Learning) on December 8, 2023 at 8:35 pm
Please correct me if I am wrong. My understanding is out of distribution could become a problem if the new data does not belong to the same probably distribution as the training data. And as a result, what fancy research on OOD are actually theatrical study of the data drift problem. Many thanks! submitted by /u/Relevant-Fortune-810 [link] [comments]
- [D] GPT-Fast performance on larger batch sizesby /u/TheMblabla (Machine Learning) on December 8, 2023 at 8:20 pm
I'm toying around with gpt-fast (https://github.com/pytorch-labs/gpt-fast) and was wondering if anyone has run experiments @ BS>1? Seems like the experiments they ran were all with a single batch. submitted by /u/TheMblabla [link] [comments]
- [P] I made a tool that automates data cleaning and corrects data entries for machine learningby /u/Apprehensive_Cut9179 (Machine Learning) on December 8, 2023 at 6:55 pm
Hi ML warriors, I am spending hours on my job manually adjusting data entries due to inconsistent spelling and shortcuts, so I built a tool to automate this process using large-language models. https://www.dataharmonizer.com/ The idea is to have a tool that takes in CSV exports and: > Corrects for inconsistencies in spelling (Coop vs co-op) > Harmonizes shortcuts (Limited vs Ltd.) > Corrects for spelling mistakes (serbices vs services) This is how the tool works: You can upload a CSV file and specify which row you want to extract and harmonize. The model automatically consolidates data by combining similar-looking phrases. You can edit the proposed phrase names or further consolidate entries if there are some groups the model has missed. In the end, you can download your CSV file again and push it to the database Would be great to hear some thoughts. Thanks 🙂 submitted by /u/Apprehensive_Cut9179 [link] [comments]
- [P] Face+Audio+EEG dataset for didactic purposeby /u/_link23_ (Machine Learning) on December 8, 2023 at 6:46 pm
Hello everyone, I'm a CS student and I'm trying to approach to the emotion recognition for a project. I played a little bit with this multimodal network for emotion recognition (https://github.com/katerynaCh/multimodal-emotion-recognition). I find it pretty cool, with the network that works very well with the Face+Audio modality. However, I was trying to implement in this network the emotion recognition with EEG (I don't really know how to do it, but still...) but I cannot find any dataset that contains Face, Audio and EEG data. Actually, I find the PME4 dataset (https://figshare.com/articles/dataset/PME4_Emotion_Recognition_with_Audio_Video_EEG_and_EMG/18737924, Face+Audio+EEG+EMG) but it has a very different structure than the RAVDESS dataset used for the multimodal network that I used in first place and I have no idea on how to adapt it to the network, so I was trying to find other datasets. submitted by /u/_link23_ [link] [comments]
- [R] EfficientSAM: Leveraged Masked Image Pretraining for Efficient Segment Anythingby /u/APaperADay (Machine Learning) on December 8, 2023 at 6:30 pm
Paper: https://arxiv.org/abs/2312.00863 Code: https://github.com/yformer/EfficientSAM Project page: https://yformer.github.io/efficient-sam/ Demo, models for GPU/CPU: https://huggingface.co/spaces/yunyangx/EfficientSAM Demo page 2: https://6639e86fff1fc7b618.gradio.live/ Abstract: Segment Anything Model (SAM) has emerged as a powerful tool for numerous vision applications. A key component that drives the impressive performance for zero-shot transfer and high versatility is a super large Transformer model trained on the extensive high-quality SA-1B dataset. While beneficial, the huge computation cost of SAM model has limited its applications to wider real-world applications. To address this limitation, we propose EfficientSAMs, light-weight SAM models that exhibits decent performance with largely reduced complexity. Our idea is based on leveraging masked image pretraining, SAMI, which learns to reconstruct features from SAM image encoder for effective visual representation learning. Further, we take SAMI-pretrained light-weight image encoders and mask decoder to build EfficientSAMs, and finetune the models on SA-1B for segment anything task. We perform evaluations on multiple vision tasks including image classification, object detection, instance segmentation, and semantic object detection, and find that our proposed pretraining method, SAMI, consistently outperforms other masked image pretraining methods. On segment anything task such as zero-shot instance segmentation, our EfficientSAMs with SAMI-pretrained lightweight image encoders perform favorably with a significant gain (e.g., ~4 AP on COCO/LVIS) over other fast SAM models. https://preview.redd.it/a01gzzes645c1.png?width=1211&format=png&auto=webp&s=dbc45f005da14c7655982406ff8fe83c32a954c3 submitted by /u/APaperADay [link] [comments]
- [D] A genuine and honest discussion on Collusion Ring(s)by /u/Terrible_Button_1763 (Machine Learning) on December 8, 2023 at 6:29 pm
Dear fellow NeurIPS rejects. As your deep learning, reinforcement learning, graph neural networks, and deep learning theory people fly off to New Orleans and you realize that you are left behind. I invite you to join me in this group therapy discussion, where our topic of the day is Collusion Rings. I suppose that.... the first question is do they actually exist, and what is the extent of their penetration into the machine learning academic community? As someone who struggled many years to have their first paper published, it's my anecdotal evidence that machine learning is much more about marching to the beat of the drummer, where the drummer is certainly someone who is a fan of deep learning. As someone struggling, still, to get another paper published, my anecdotal observation is the drum beating has gotten even more fierce over the last few years. As someone who has had many, many conversations with others whom are also marginalized, our anecdotal data pools not quite to a dataset, but a filtration which is not i.i.d., but certainly indicates actively farming deep learning citations is the better choice for our careers. As someone currently reviewing for ICLR/AAAI/AISTATS. My anecdotal evidence is the reviewer coordination is with secret handshakes, keywords, citations, reference lists, topics, arxiv preprints, and shibboleths. I hope you find the bravery to share your experience as one who is on the inside looking out, or on the outside looking in. As a beacon of hope, I remind you to read The Revolution Hasn't Happened Yet by Michael Jordan. As a final question to ponder. Is the deep learning collusion ring already collapsing, and will it collapse further? submitted by /u/Terrible_Button_1763 [link] [comments]
- [R] Recurrent Distance-Encoding Neural Networks for Graph Representation Learningby /u/APaperADay (Machine Learning) on December 8, 2023 at 6:05 pm
arXiv: https://arxiv.org/abs/2312.01538 OpenReview: https://openreview.net/forum?id=lNIj5FdXsC Abstract: Graph neural networks based on iterative one-hop message passing have been shown to struggle in harnessing information from distant nodes effectively. Conversely, graph transformers allow each node to attend to all other nodes directly, but suffer from high computational complexity and have to rely on ad-hoc positional encoding to bake in the graph inductive bias. In this paper, we propose a new architecture to reconcile these challenges. Our approach stems from the recent breakthroughs in long-range modeling provided by deep state-space models on sequential data: for a given target node, our model aggregates other nodes by their shortest distances to the target and uses a parallelizable linear recurrent network over the chain of distances to provide a natural encoding of its neighborhood structure. With no need for positional encoding, we empirically show that the performance of our model is highly competitive compared with that of state-of-the-art graph transformers on various benchmarks, at a drastically reduced computational complexity. In addition, we show that our model is theoretically more expressive than one-hop message passing neural networks. https://preview.redd.it/wf256ehd245c1.png?width=937&format=png&auto=webp&s=3d3d15544dafdc1c562694a0336448652cebd753 submitted by /u/APaperADay [link] [comments]
- [D] Class-Discriminative Attention Maps for Vision Transformersby /u/wro_o (Machine Learning) on December 8, 2023 at 3:38 pm
Hi gentlepeople, I just posted this preprint on arxiv, and trying to think where to submit. I would absolutely love to hear your feedback. Usually I dont post it here, but I thought this is really interesting and broadly useful, so I'm trying to aim higher. Lmk! Basically, we proposed Class-Discriminative Attention Maps (CDAM) for Vision Transformers. CDAM is a heat map (also called a saliency or relevance map) of how important each pixel is with respect to the selected class in ViT models. CDAM retains the advantages of attention maps (high quality semantic segmentation), while being class discriminative and providing implicit regularization. Moreover, you don't even have to build a classifier on ViT. You can simply select a few images sharing a common object ("concept"), and CDAM will explain that. Live demo (upload your images): https://cdam.informatism.com/ Check out the arxiv: https://arxiv.org/abs/2312.02364 Python/pytorch implementation: https://github.com/lenbrocki/CDAM https://preview.redd.it/txq752hyb35c1.png?width=2327&format=png&auto=webp&s=9dddb8f21521d984020eedb3a1241ca8e5c6076f https://preview.redd.it/nwbft1hyb35c1.png?width=1400&format=png&auto=webp&s=26796e3f847ca5e247f9bd2a6ad8e4f538adf058 submitted by /u/wro_o [link] [comments]
- [D] In Neural Networks is it effective to select the best-performing output neurons to train the model?by /u/mbrostami (Machine Learning) on December 8, 2023 at 12:29 pm
Imagine I have M number of inputs and 1 target. But instead of having 1 neuron in output layer in model, I put 1000 neurones in output layer with a custom loss function that calculates loss of each individual output with target in a batch size of X. Then I can find the minimum mean loss of each output in a batch. Then will back-propagate based on minimum loss. In next epoch I’ll keep the weights of the selected neuron, but will randomise the weights of other output neurones. Is this something that might end up useful and faster to converge, or am I missing something? submitted by /u/mbrostami [link] [comments]
- [R] Free3D: Consistent Novel View Synthesis without 3D Representationby /u/lyndonzheng (Machine Learning) on December 8, 2023 at 10:59 am
Paper: https://arxiv.org/pdf/2312.04551.pdf Webpage: https://chuanxiaz.com/free3d/ Video: https://youtu.be/7CdYuZ7D1DY https://preview.redd.it/pop4hmkey15c1.png?width=892&format=png&auto=webp&s=15fb0f2ec671224133b1e99728f80fce5da9048c submitted by /u/lyndonzheng [link] [comments]
- [D] Is masters while working full time as ML engineer worth it?by /u/Fluid-Pipe-2831 (Machine Learning) on December 8, 2023 at 5:52 am
I’m currently a first year graduate from undergraduate college working full time as an ML engineer. My company offers to pay for a graduate degree and I’m really considering getting a masters. First question: for those who are ML or data scientists, what were your degrees in? Second question: how much of this will suck? Is taking two classes per semester reasonable? And are online universities considered okay? Third question: how much of a pay bump did you see after getting your masters? submitted by /u/Fluid-Pipe-2831 [link] [comments]
- [D] Advice on First ML Research Paperby /u/waffleman221 (Machine Learning) on December 8, 2023 at 3:43 am
Hi guys, I’m in my third year of university and I’m currently working on my first research paper. The project explores the trade-offs in accuracy, interpretability and computational complexity if NAS created DNNs. Similar work has been done within the past 5 years, but they only explore creating and using computer vision benchmarks like NAS-Bench 201. These benchmarks create a search space of convolution networks but none only explore DNNs, RNNs at all. My work explores only DNNNs without using an existing benchmark and I wrote in the further work that developing a standard non-convolution benchmark can be done. I feel like my results can help research in this particular area since a lot of people value the use of DNNs. I want to submit it, but I would want to submit it at at least a medium-tier conference but I’m so insecure about my work. Any advice on how I should tackle this? Also, I did this project 100% on my own which is why I am even more embarrassed. Feel like it’s crappy and I’m wasting my time submitted by /u/waffleman221 [link] [comments]
- [D] Thoughts on Mamba?by /u/ExaminationNo8522 (Machine Learning) on December 7, 2023 at 9:29 pm
I ran the NanoGPT of Karparthy replacing Self-Attention with Mamba on his TinyShakespeare Dataset and within 5 minutes it started spitting out the following: https://preview.redd.it/4r96tp6lxx4c1.png?width=836&format=png&auto=webp&s=10f2f61cd4cea96f4f903cb2070835fc5d1df951 https://preview.redd.it/32ler5vnxx4c1.png?width=622&format=png&auto=webp&s=dd00e53f43dd0afa058758a987901ee6789d2258 https://preview.redd.it/sc96i4xoxx4c1.png?width=678&format=png&auto=webp&s=94d2ed279054363d3ed2b6beed65be89468582b0 So much faster than self-attention, and so much smoother, running at 6 epochs per second. I'm honestly gobsmacked. https://colab.research.google.com/drive/1g9qpeVcFa0ca0cnhmqusO4RZtQdh9umY?usp=sharing Some loss graphs: Multihead attention without truncation(x is iterations in 10s, and y is loss) Multihead attention with truncation(x is iterations in 10s, and y is loss) Mamba loss graph(x is iterations in 10s, and y is loss) submitted by /u/ExaminationNo8522 [link] [comments]
- Getir end-to-end workforce management: Amazon Forecast and AWS Step Functionsby Nafi Ahmet Turgut (AWS Machine Learning Blog) on December 7, 2023 at 4:16 pm
This is a guest post co-authored by Nafi Ahmet Turgut, Mehmet İkbal Özmen, Hasan Burak Yel, Fatma Nur Dumlupınar Keşir, Mutlu Polatcan and Emre Uzel from Getir. Getir is the pioneer of ultrafast grocery delivery. The technology company has revolutionized last-mile delivery with its grocery in-minutes delivery proposition. Getir was founded in 2015 and operates
- Mitigate hallucinations through Retrieval Augmented Generation using Pinecone vector database & Llama-2 from Amazon SageMaker JumpStartby Vedant Jain (AWS Machine Learning Blog) on December 6, 2023 at 7:41 pm
Despite the seemingly unstoppable adoption of LLMs across industries, they are one component of a broader technology ecosystem that is powering the new AI wave. Many conversational AI use cases require LLMs like Llama 2, Flan T5, and Bloom to respond to user queries. These models rely on parametric knowledge to answer questions. The model
- Techniques for automatic summarization of documents using language modelsby Nick Biso (AWS Machine Learning Blog) on December 6, 2023 at 3:14 pm
Summarization is the technique of condensing sizable information into a compact and meaningful form, and stands as a cornerstone of efficient communication in our information-rich age. In a world full of data, summarizing long texts into brief summaries saves time and helps make informed decisions. Summarization condenses content, saving time and improving clarity by presenting
- Boosting RAG-based intelligent document assistants using entity extraction, SQL querying, and agents with Amazon Bedrockby Mohamed Ali Jamaoui (AWS Machine Learning Blog) on December 6, 2023 at 3:12 pm
Conversational AI has come a long way in recent years thanks to the rapid developments in generative AI, especially the performance improvements of large language models (LLMs) introduced by training techniques such as instruction fine-tuning and reinforcement learning from human feedback. When prompted correctly, these models can carry coherent conversations without any task-specific training data.
- How Q4 Inc. used Amazon Bedrock, RAG, and SQLDatabaseChain to address numerical and structured dataset challenges building their Q&A chatbotby Stanislav Yeshchenko (AWS Machine Learning Blog) on December 6, 2023 at 3:10 pm
This post is co-written with Stanislav Yeshchenko from Q4 Inc. Enterprises turn to Retrieval Augmented Generation (RAG) as a mainstream approach to building Q&A chatbots. We continue to see emerging challenges stemming from the nature of the assortment of datasets available. These datasets are often a mix of numerical and text data, at times structured,
- Enable faster training with Amazon SageMaker data parallel libraryby Apoorv Gupta (AWS Machine Learning Blog) on December 5, 2023 at 6:16 pm
Large language model (LLM) training has become increasingly popular over the last year with the release of several publicly available models such as Llama2, Falcon, and StarCoder. Customers are now training LLMs of unprecedented size ranging from 1 billion to over 175 billion parameters. Training these LLMs requires significant compute resources and time as hundreds
- Use custom metadata created by Amazon Comprehend to intelligently process insurance claims using Amazon Kendraby Amit Chaudhary (AWS Machine Learning Blog) on December 5, 2023 at 6:04 pm
Structured data, defined as data following a fixed pattern such as information stored in columns within databases, and unstructured data, which lacks a specific form or pattern like text, images, or social media posts, both continue to grow as they are produced and consumed by various organizations. For instance, according to International Data Corporation (IDC),
- Foundational data protection for enterprise LLM acceleration with Protopia AIby Balaji Chandrasekaran (AWS Machine Learning Blog) on December 5, 2023 at 5:36 pm
The post describes how you can overcome the challenges of retaining data ownership and preserving data privacy while using LLMs by deploying Protopia AI’s Stained Glass Transform to protect your data. Protopia AI has partnered with AWS to deliver the critical component of data protection and ownership for secure and efficient enterprise adoption of generative AI. This post outlines the solution and demonstrates how it can be used in AWS for popular enterprise use cases like Retrieval Augmented Generation (RAG) and with state-of-the-art LLMs like Llama 2.
- How Getir reduced model training durations by 90% with Amazon SageMaker and AWS Batchby Nafi Ahmet Turgut (AWS Machine Learning Blog) on December 4, 2023 at 2:57 pm
This is a guest post co-authored by Nafi Ahmet Turgut, Hasan Burak Yel, and Damla Şentürk from Getir. Established in 2015, Getir has positioned itself as the trailblazer in the sphere of ultrafast grocery delivery. This innovative tech company has revolutionized the last-mile delivery segment with its compelling offering of “groceries in minutes.” With a
- [D] Simple Questions Threadby /u/AutoModerator (Machine Learning) on December 3, 2023 at 4:00 pm
Please post your questions here instead of creating a new thread. Encourage others who create new posts for questions to post here instead! Thread will stay alive until next one so keep posting after the date in the title. Thanks to everyone for answering questions in the previous thread! submitted by /u/AutoModerator [link] [comments]
- Boosting developer productivity: How Deloitte uses Amazon SageMaker Canvas for no-code/low-code machine learningby Chida Sadayappan (AWS Machine Learning Blog) on December 1, 2023 at 8:40 pm
The ability to quickly build and deploy machine learning (ML) models is becoming increasingly important in today’s data-driven world. However, building ML models requires significant time, effort, and specialized expertise. From data collection and cleaning to feature engineering, model building, tuning, and deployment, ML projects often take months for developers to complete. And experienced data
- Experience the new and improved Amazon SageMaker Studioby Mair Hasco (AWS Machine Learning Blog) on December 1, 2023 at 4:04 pm
Launched in 2019, Amazon SageMaker Studio provides one place for all end-to-end machine learning (ML) workflows, from data preparation, building and experimentation, training, hosting, and monitoring. As we continue to innovate to increase data science productivity, we’re excited to announce the improved SageMaker Studio experience, which allows users to select the managed Integrated Development Environment (IDE)
- Amazon SageMaker simplifies setting up SageMaker domain for enterprises to onboard their users to SageMakerby Ozan Eken (AWS Machine Learning Blog) on December 1, 2023 at 4:01 pm
As organizations scale the adoption of machine learning (ML), they are looking for efficient and reliable ways to deploy new infrastructure and onboard teams to ML environments. One of the challenges is setting up authentication and fine-grained permissions for users based on their roles and activities. For example, MLOps engineers typically perform model deployment activities,
- Welcome to a New Era of Building in the Cloud with Generative AI on AWSby Swami Sivasubramanian (AWS Machine Learning Blog) on November 30, 2023 at 10:36 pm
We believe generative AI has the potential over time to transform virtually every customer experience we know. The number of companies launching generative AI applications on AWS is substantial and building quickly, including adidas, Booking.com, Bridgewater Associates, Clariant, Cox Automotive, GoDaddy, and LexisNexis Legal & Professional, to name just a few. Innovative startups like Perplexity
- Package and deploy classical ML and LLMs easily with Amazon SageMaker, part 2: Interactive User Experiences in SageMaker Studioby Raghu Ramesha (AWS Machine Learning Blog) on November 30, 2023 at 8:45 pm
Amazon SageMaker is a fully managed service that enables developers and data scientists to quickly and easily build, train, and deploy machine learning (ML) models at scale. SageMaker makes it easy to deploy models into production directly through API calls to the service. Models are packaged into containers for robust and scalable deployments. SageMaker provides
- Package and deploy classical ML and LLMs easily with Amazon SageMaker, part 1: PySDK Improvementsby Melanie Li (AWS Machine Learning Blog) on November 30, 2023 at 8:43 pm
Amazon SageMaker is a fully managed service that enables developers and data scientists to quickly and effortlessly build, train, and deploy machine learning (ML) models at any scale. SageMaker makes it straightforward to deploy models into production directly through API calls to the service. Models are packaged into containers for robust and scalable deployments. Although
- New – Code Editor, based on Code-OSS VS Code Open Source now available in Amazon SageMaker Studioby Eric Peña (AWS Machine Learning Blog) on November 30, 2023 at 6:30 pm
Today, we are excited to announce support for Code Editor, a new integrated development environment (IDE) option in Amazon SageMaker Studio. Code Editor is based on Code-OSS, Visual Studio Code Open Source, and provides access to the familiar environment and tools of the popular IDE that machine learning (ML) developers know and love, fully integrated
- Scale foundation model inference to hundreds of models with Amazon SageMaker – Part 1by Mehran Najafi (AWS Machine Learning Blog) on November 30, 2023 at 6:18 pm
As democratization of foundation models (FMs) becomes more prevalent and demand for AI-augmented services increases, software as a service (SaaS) providers are looking to use machine learning (ML) platforms that support multiple tenants—for data scientists internal to their organization and external customers. More and more companies are realizing the value of using FMs to generate
- Reduce model deployment costs by 50% on average using the latest features of Amazon SageMakerby James Park (AWS Machine Learning Blog) on November 30, 2023 at 6:04 pm
As organizations deploy models to production, they are constantly looking for ways to optimize the performance of their foundation models (FMs) running on the latest accelerators, such as AWS Inferentia and GPUs, so they can reduce their costs and decrease response latency to provide the best experience to end-users. However, some FMs don’t fully utilize
- Minimize real-time inference latency by using Amazon SageMaker routing strategiesby James Park (AWS Machine Learning Blog) on November 30, 2023 at 6:02 pm
Amazon SageMaker makes it straightforward to deploy machine learning (ML) models for real-time inference and offers a broad selection of ML instances spanning CPUs and accelerators such as AWS Inferentia. As a fully managed service, you can scale your model deployments, minimize inference costs, and manage your models more effectively in production with reduced operational
- Build and evaluate machine learning models with advanced configurations using the SageMaker Canvas model leaderboardby Janisha Anand (AWS Machine Learning Blog) on November 30, 2023 at 5:50 pm
Amazon SageMaker Canvas is a no-code workspace that enables analysts and citizen data scientists to generate accurate machine learning (ML) predictions for their business needs. Starting today, SageMaker Canvas supports advanced model build configurations such as selecting a training method (ensemble or hyperparameter optimization) and algorithms, customizing the training and validation data split ratio, and
A Twitter List by enoumen