I’m currently a full stack software engineer with 1 YOE looking to become a Machine Learning Engineer. If you work full time as an MLE or if you’re someone who knows a lot about this industry, what tips do you have for me? Would especially like to know what tools/skills I should acquire to qualify for the job. For more context: I’m trying to transition within the next year. submitted by /u/kiqoo [link] [comments]

Hello all, I am starting a golf apparel company. I am a bioinformatician by training with years of experience. I have familiarity with R and Python. My question to you all is what models should I consider and how should I implement them for forecasting sales data and building predictive models? Any GitHub links or tutorials, training materials, etc. would be appreciated! Thank you! submitted by /u/docdropz [link] [comments]

What's the process/approach:architecture of combining multiple models of different modality to generate sane output? Just curious what your experiences have been. Any pointers or links to research would be handy. submitted by /u/hophophop1233 [link] [comments]

I'm quite newbie so forgive me if anything I say or ask might be dumb. I was wondering some things why todays cnn (like yolov8/9) are faster even if theoretically transformer have the same complexity. Are there SMoE like Mixtral in ViT I have some paper but they do not seem to have brought some significant improvement, am I missing something? On practical term even considering nlp small SMoE work in practice according to your experince? submitted by /u/mitadev [link] [comments]

I've been examining the real-world context limits of large language models (LLMs), and I wanted to share some enlightening findings from a recent benchmark (RULER) that cuts through the noise. What’s the RULER Benchmark? Developed by NVIDIA, RULER is a benchmark designed to test LLMs' ability to handle long-context information. It's more intricate than the common retrieval-focused NIAH benchmark. RULER evaluates models based on their performance in understanding and using longer pieces of text. Table highlighting RULER benchmark results and effective context lengths of leading LLMs Performance Highlights from the Study: Llama2-7B (chat): Shows decent initial performance but doesn't sustain at higher context lengths. GPT-4: Outperforms others significantly, especially at greater lengths of context, maintaining above 80% accuracy. Command-R (35B): Performs comparably well, slightly behind GPT-4. Yi (34B): Shows strong performance, particularly up to 32K context length. Mixtral (8x7B): Similar to Yi, holds up well until 32K context. Mistral (7B): Drops off in performance as context increases, more so after 32K. ChatGLM (6B): Struggles with longer contexts, showing a steep decline. LWM (7B): Comparable to ChatGLM, with a noticeable decrease in longer contexts. Together (7B): Faces difficulties maintaining accuracy as context length grows. LongChat (13B): Fares reasonably up to 4K but drops off afterwards. LongAlpaca (13B): Shows the most significant drop in performance as context lengthens. Key Takeaways: All models experience a performance drop as the context length increases, without exception. The claimed context length by LLMs often doesn't translate into effective processing ability at those lengths. GPT-4 emerges as a strong leader but isn't immune to decreased accuracy at extended lengths. Why Does This Matter? As AI developers, it’s critical to look beyond the advertised capabilities of LLMs. Understanding the effective context length can help us make informed decisions when integrating these models into applications. What's Missing in the Evaluation? Notably, Google’s Gemini and Claude 3 were not part of the evaluated models. RULER is now open-sourced, paving the way for further evaluations and transparency in the field. Sources I recycled a lot of this (and tried to make it more digestible and easy to read) from the following post, further sources available here: Harmonious.ai Weekly paper roundup: RULER: real context size of LLMs (4/8/2024) submitted by /u/ParsaKhaz [link] [comments]

Hello Reddit! We're a group of students from Carnegie Mellon passionate about data. We've been working on a project that tackles one of the biggest challenges in AI today: data bias. Our tool is designed to audit your datasets and generate synthetic, privacy-safe data that helps promote fairness in AI applications. Here’s why we think it could be a game-changer: Detect and Correct Data Biases: Ensures your AI models are built on fair and balanced data. Enhance Data Quality: Improves the reliability and performance of your AI systems. Generate Synthetic Data: Expands your dataset without compromising privacy. We believe in a future where data-driven technologies are equitable and just, and we're eager to contribute to making that a reality. We’d love to get your insights and feedback to further refine our tool. Join Us in Shaping the Future of Fair AI Are you interested in AI fairness? Do you have experiences with biased data or ideas on how to address these issues? Let’s start a conversation! Comment here or send us a DM. We’re here to discuss and collaborate on getting you the data you deserve! submitted by /u/ComplexAnalysis42 [link] [comments]

Hi Everyone, There is a problem we are trying to solve and I want to know if there is particular approach/algorithm which will work the team. We are using Cloudera Hadoop system at our company and currently we have hard coded rule system to trigger if there is missing data on different dimensions, like time and different configuration items. We are trying to reduce the number of incidents based on what our past observations which we can code it up easily by writing more rules to group the common issues. But at the same time we were wondering if there is an algo/technique which will help us group the common issues based on feature similarities, because the features which we are forming rules by writing common code might become outdated and we soon be writing more rules. Based on my previous beginner level knowledge of ML seems like it is clustering problem where we don't know the number of clusters and some google and chatgpting seems like DBSCAN will fit but don't want to spend lot of time trying out and finding out this is not the right approach to solve the problem. Anyone tried to solve a similar problem and what approach did they take? Anyone has some guidance on this? Features which we have are: time (through, we don't rely on that much and issues at different time are considered different incidents, so may not be that important and would need to give less weight?) 2.cluster Market data pipeline lineage (part of it is in place and part of it is in data collection stage, I believe this will make the coding part complicated if we go with rule based system) Product/Service affected Configuration item Thanks appreciate any help in advance. (let me know if this is not the correct sub for this type of question) submitted by /u/Abject-Log-1249 [link] [comments]

I built an embedding visualizer that runs entirely in the browser, using Pyodide and a marimo notebook: https://marimo.app/l/55kclj ​ The notebook uses PCA to project the UCI digits dataset to 2D, then uses marimo's dataflow execution engine to automatically wire plot selections to Python. Select data in the scatterplot, view the original images. ​ The example uses PCA because it can be computed entirely in the browser, but the notebook can be used as a template to explore a scatterplot produced by any other means. submitted by /u/akshayka [link] [comments]

I'm trying to develop a 2D CNN for a time series model and am having a bit of trouble deciding/understanding what the input shape of the 2D CNN should be. The data I have consists of 7 time series, each of which has 5 features, one target (response) variable and 60 time steps. So for example, the data for one time series would look like this (for the first 3 rows): Timestamp, feature1, feature2, feature3 ... target 06/01/2012 01:00:00, 500, 29, 0.3 ... 33 06/01/2012 02:00:00, 200, 32, 0.1 ... 45 06/01/2012 03:00:00, 600, 23, 0.24 ... 56 My goal is to have the CNN be trained on the entirety of 6 of the time series and tested on the last one. I did a bit of research online and found people using the input shape as (number of samples, number of time series, window length, number of features). My questions are: what would the dimensions of my X_train, X_test, y_train, y_test be? if this input shape is valid, why? submitted by /u/redreaper71_ [link] [comments]

I'm looking for a dataset consisting of hi-resolution and zoomed in SEM images of computer chips (Can be any chip or IC) along with their idealized layout images (can be rgb or grayscale). Basically I want one sample of the dataset to contain a pair of images (1) the SEM image and (2) the idealized layout corresponding to that SEM image. Does any of you know if a dataset like this exists? Doesn't matter how small at this point I'll take anything. I've been looking for a some time now and can't find anything. Not sure if I'm searching for the wrong thing. https://preview.redd.it/o0srhuu0havc1.png?width=243&format=png&auto=webp&s=8805b9967062a674f13658c727042d69955a567b https://preview.redd.it/q5ysnhu2havc1.png?width=240&format=png&auto=webp&s=c432fd07beab27e261c4579b970c64faebbd049d submitted by /u/budgetfroot [link] [comments]

Was super excited to read the news from Meta this morning, particularly around the HumanEval scores the 70B model got. Thought it'd be useful to make the new Llama 3 70B available to anyone that wants to try it, so I added it to my VS Code coding copilot extension double.bot. Also making it free for the first 50 messages so everyone gets a chance to try it while we wait for the quantized versions to run locally submitted by /u/geepytee [link] [comments]

Llama-3 is making waves in the AI community. I was curious how it will perform in the medical domain, Here are the evaluation results for Llama-3 (7B and 70B) on a medical domain benchmark consisting of 9 diverse datasets ​ https://preview.redd.it/t7cqlkyt9avc1.png?width=1474&format=png&auto=webp&s=bb053d032093dd09c892a3fb570b3eb40a8a288c I'll be fine-tuning, evaluating & releasing Llama-3 & different LLMs over the next few days on different Medical and Legal benchmarks. Follow the updates here: https://twitter.com/aadityaura ​ https://preview.redd.it/9egbcayv9avc1.png?width=1344&format=png&auto=webp&s=436a972421d5568e1a544962b8cfd1c7b14efe04 ​ ​ submitted by /u/aadityaura [link] [comments]

I have been hunting jobs for almost 4 months now. It was after 2 years, that I opened my eyes to the outside world and in the beginning, the world fell apart because I wasn't aware of how much the industry has changed and genAI and LLMs were now mandatory things. Before, I was just limited to using chatGPT as UI. So, after preparing for so many months it felt as if I was walking in circles and running across here and there without an in-depth understanding of things. I went through around 40+ job posts and studied their requirements, (for a medium seniority DS position). So, I created a plan and then worked on each task one by one. Here, if anyone is interested, you can take a look at the important tools and libraries, that are relevant for the job hunt. Github, Notion I am open to your suggestions and edits, Happy preparation! submitted by /u/xandie985 [link] [comments]

https://github.com/Never-Over/chatpdb ChatGPT can be an awesome tool for quickly diagnosing issues from error output, but pasting snippets into the chat window often misses important context and forces you to switch focus and lose your flow. That’s why we built chatpdb - a python debugger with ChatGPT built in. Simply use chatpdb like you would use ipdb or pdb; it’s a drop in replacement with the exact same functionality. The only addition is the new y keyword, which will trigger a response from ChatGPT. Here’s how it works: > /Programming/test-chatpdb/lib.py(2)echo_platform() 1 def echo_platform(platform: str): ----> 2 print("You are running on:" + platform) 3 ipdb> y The exception occurred because the function `echo_platform` tries to concatenate the string "You are running on:" with the `platform` variable, which is `None`. [...] In this example, chatpdb correctly diagnoses an error in the current function as being caused by a bug in the calling function. chatpdb automatically passes relevant context about your program, including the source code, stack trace, and error information if available to ChatGPT. If you have a more specific question you can also supply a prompt to y: > /Programming/test-chatpdb/lib.py(2)echo_platform() 1 def echo_platform(platform: str): ----> 2 print("You are running on:" + platform) 3 ipdb> y "Why is platform coming through as None?" The variable `platform` is coming through as `None` because the environment variable `"PLATFORM"` is not set in your system's environment variables. [...] It’s easy to install and set up: pip install chatpdb export OPENAI_API_KEY=.... import chatpdb chatpdb.set_trace() chatpdb meets you where you are – AI tooling that’s only invoked when you need it. We hope you give it a try! We’d love any feedback or suggestions. Docs submitted by /u/the1024 [link] [comments]

As you navigate the complexities of cloud migration, the need for a structured, secure, and compliant environment is paramount. AWS Landing Zone addresses this need by offering a standardized approach to deploying AWS resources. This makes sure your cloud foundation is built according to AWS best practices from the start. With AWS Landing Zone, you eliminate the guesswork in security configurations, resource provisioning, and account management. It’s particularly beneficial for organizations looking to scale without compromising on governance or control, providing a clear path to a robust and efficient cloud setup. In this post, we show you how to generate customized, compliant IaC scripts for AWS Landing Zone using Amazon Bedrock.

When are the ICML meta reviews releasing? Will it be declared along with the final paper notification? The webpage says its due 16th April. https://icml.cc/Conferences/2024/ReviewerInstructions submitted by /u/Personal_Click_6502 [link] [comments]

You’ve likely experienced the challenge of taking notes during a meeting while trying to pay attention to the conversation. You’ve probably also experienced the need to quickly fact-check something that’s been said, or look up information to answer a question that’s just been asked in the call. Or maybe you have a team member that always joins meetings late, and expects you to send them a quick summary over chat to catch them up. Then there are the times that others are talking in a language that’s not your first language, and you’d love to have a live translation of what people are saying to make sure you understand correctly. And after the call is over, you usually want to capture a summary for your records, or to send to the participants, with a list of all the action items, owners, and due dates. All of this, and more, is now possible with our newest sample solution, Live Meeting Assistant (LMA).

Paper: https://arxiv.org/abs/2311.09480 Tweet: https://x.com/NickLourie/status/1770077925779337563 Code: https://github.com/nicholaslourie/opda Docs: https://nicholaslourie.github.io/opda/tutorial/usage.html Abstract: The choice of hyperparameters greatly impacts performance in natural language processing. Often, it is hard to tell if a method is better than another or just better tuned. Tuning curves fix this ambiguity by accounting for tuning effort. Specifically, they plot validation performance as a function of the number of hyperparameter choices tried so far. While several estimators exist for these curves, it is common to use point estimates, which we show fail silently and give contradictory results when given too little data. Beyond point estimates, confidence bands are necessary to rigorously establish the relationship between different approaches. We present the first method to construct valid confidence bands for tuning curves. The bands are exact, simultaneous, and distribution-free, thus they provide a robust basis for comparing methods. Empirical analysis shows that while bootstrap confidence bands, which serve as a baseline, fail to approximate their target confidence, ours achieve it exactly. We validate our design with ablations, analyze the effect of sample size, and provide guidance on comparing models with our method. To promote confident comparisons in future work, we release opda: an easy-to-use library that you can install with pip. submitted by /u/nicholaslourie [link] [comments]

https://preview.redd.it/fh44g3n4m9vc1.png?width=1383&format=png&auto=webp&s=9b3e499bd51aeb10559f4636eba2a1677d4a08a3 InternVL is a multi-modal foundation model, which is accepted as an Oral paper for CVPR 2024. The latest version InternVL v1.5 ranks first in the OpenCompass multi-modal model benchmark. Demo： https://internvl.opengvlab.com/ Model Download: https://huggingface.co/collections/OpenGVLab/internvl-65b92d6be81c86166ca0dde4 OpenCompass： https://rank.opencompass.org.cn Some examples: https://preview.redd.it/rwj7vs9rm9vc1.jpg?width=902&format=pjpg&auto=webp&s=514e14e692db8ea7bd5a66cc36b1ca3f8351102c https://preview.redd.it/vtwjml3qm9vc1.png?width=2508&format=png&auto=webp&s=e32c044d4bc60ef28baf64dccdcb5fe9b10dfc61 https://preview.redd.it/p51vt3xpn9vc1.png?width=2609&format=png&auto=webp&s=73907e5ffb4d9b9bd4250cbce53e3bd29dedabf1 submitted by /u/flyforlight [link] [comments]

Today, we are excited to announce that Meta Llama 3 foundation models are available through Amazon SageMaker JumpStart to deploy and run inference. The Llama 3 models are a collection of pre-trained and fine-tuned generative text models. In this post, we walk through how to discover and deploy Llama 3 models via SageMaker JumpStart. What is

https://llama.meta.com/llama3/ ​ ​ https://preview.redd.it/n3lwb4xfj9vc1.png?width=3840&format=png&auto=webp&s=b756d89c50c627955668d5ac16df82f7af01cdbc submitted by /u/we_are_mammals [link] [comments]

Paper: https://arxiv.org/abs/2403.12037 Code: https://github.com/Zhoues/MineDreamer Models and Dataset: https://huggingface.co/Zhoues Abstract: It is a long-lasting goal to design a generalist-embodied agent that can follow diverse instructions in human-like ways. However, existing approaches often fail to steadily follow instructions due to difficulties in understanding abstract and sequential natural language instructions. To this end, we introduce MineDreamer, an open-ended embodied agent built upon the challenging Minecraft simulator with an innovative paradigm that enhances instruction-following ability in low-level control signal generation. Specifically, MineDreamer is developed on top of recent advances in Multimodal Large Language Models (MLLMs) and diffusion models, and we employ a Chain-of-Imagination (CoI) mechanism to envision the step-by-step process of executing instructions and translating imaginations into more precise visual prompts tailored to the current state; subsequently, the agent generates keyboard-and-mouse actions to efficiently achieve these imaginations, steadily following the instructions at each step. Extensive experiments demonstrate that MineDreamer follows single and multi-step instructions steadily, significantly outperforming the best generalist agent baseline and nearly doubling its performance. Moreover, qualitative analysis of the agent's imaginative ability reveals its generalization and comprehension of the open world. submitted by /u/SeawaterFlows [link] [comments]

Paper: https://arxiv.org/abs/2404.09937 Code: https://github.com/hkust-nlp/llm-compression-intelligence Datasets: https://huggingface.co/datasets/hkust-nlp/llm-compression Abstract: There is a belief that learning to compress well will lead to intelligence. Recently, language modeling has been shown to be equivalent to compression, which offers a compelling rationale for the success of large language models (LLMs): the development of more advanced language models is essentially enhancing compression which facilitates intelligence. Despite such appealing discussions, little empirical evidence is present for the interplay between compression and intelligence. In this work, we examine their relationship in the context of LLMs, treating LLMs as data compressors. Given the abstract concept of "intelligence", we adopt the average downstream benchmark scores as a surrogate, specifically targeting intelligence related to knowledge and commonsense, coding, and mathematical reasoning. Across 12 benchmarks, our study brings together 30 public LLMs that originate from diverse organizations. Remarkably, we find that LLMs' intelligence -- reflected by average benchmark scores -- almost linearly correlates with their ability to compress external text corpora. These results provide concrete evidence supporting the belief that superior compression indicates greater intelligence. Furthermore, our findings suggest that compression efficiency, as an unsupervised metric derived from raw text corpora, serves as a reliable evaluation measure that is linearly associated with the model capabilities. We open-source our compression datasets as well as our data collection pipelines to facilitate future researchers to assess compression properly. submitted by /u/SeawaterFlows [link] [comments]

We are an AI consultancy and this happens with us over and over again... We start a new LLM project with a client. Their engineers get to 80% super quickly. They have a lot of edge cases and want us to finish up the remaining 20%. We ask them about evals. And of course they don't have them. We create the eval framework, improve the pipeline iteratively, and voila. Job done, everybody is happy. I seriously think that based on what we see the best AI product teams are going to be the ones that spend an absurd amount of time on evals. It's boring, it's repetitive but it makes the difference between an amazing AI product and an underperforming one. submitted by /u/BootstrapGuy [link] [comments]

Each text has no more than 15 words, and the classes are highly imbalanced. But they all have at least 30 or so instances. I was successful with data of the same nature but with around 15 labels with an ensemble of gradient boosting models. Before diving into testing a bunch models I wondered if there’s some strategy to tackle high-dimensional problems like this one. Some problems are just not solvable, let’s face it. But what would you guys try? submitted by /u/Melodic_Reality_646 [link] [comments]

Hello, I am training a model on tabular data which has already been preprocessed (scaled, PCA). Currently there are over 50k rows and 10 columns. The loss are high, not sure where I'm doing wrong For context, I'm using MSE as my loss function, 0.01 learning rate and 256 batch size. Thank you so much. This is how my init code looks like: class NN(nn.Module): def init(self): super(NN, self).init() # Tabular data processing layers self.fc1 = nn.Linear(10, 64) self.fc2 = nn.Linear(64, 32) self.fc3 = nn.Linear(32, 16) self.fc4 = nn.Linear(16, 1) self.bn1 = nn.BatchNorm1d(64) self.bn2 = nn.BatchNorm1d(32) self.bn3 = nn.BatchNorm1d(16) self.relu = nn.ReLU() self.dropout = nn.Dropout(0.25) def forward(self, x_tab, x_img): out = self.fc1(x_tab) out = self.bn1(out) out = self.relu(out) out = self.dropout(out) out = self.fc2(out) out = self.bn2(out) out = self.relu(out) out = self.dropout(out) out = self.fc3(out) out = self.bn3(out) out = self.relu(out) out = self.dropout(out) out = self.fc4(out) return out Output: Epoch 1/30, Loss: 16834.8088 Epoch 2/30, Loss: 4379.7037 Epoch 3/30, Loss: 3361.2462 Epoch 4/30, Loss: 3255.9039 Epoch 5/30, Loss: 3255.8603 Epoch 6/30, Loss: 3243.9488 Epoch 7/30, Loss: 3235.4387 Epoch 8/30, Loss: 3213.4688 Epoch 9/30, Loss: 3189.1130 Epoch 10/30, Loss: 3174.2118 Epoch 11/30, Loss: 3168.1597 Epoch 12/30, Loss: 3155.3225 Epoch 13/30, Loss: 3150.0659 Epoch 14/30, Loss: 3119.2989 Epoch 15/30, Loss: 3117.0893 Epoch 16/30, Loss: 3130.4699 Epoch 17/30, Loss: 3126.7107 Epoch 18/30, Loss: 3110.9422 Epoch 19/30, Loss: 3119.8601 Epoch 20/30, Loss: 3094.5037 Epoch 21/30, Loss: 3054.4725 Epoch 22/30, Loss: 3079.4411 Epoch 23/30, Loss: 3064.4010 Epoch 24/30, Loss: 3049.7988 Epoch 25/30, Loss: 3022.9714 Epoch 26/30, Loss: 3029.0342 Epoch 27/30, Loss: 3034.8153 Epoch 28/30, Loss: 3025.2383 Epoch 29/30, Loss: 3052.9892 Epoch 30/30, Loss: 3033.2717 submitted by /u/sparttann [link] [comments]

submitted by /u/TheMrZZ0 [link] [comments]

We are excited to announce that Slack, a Salesforce company, has collaborated with Amazon SageMaker JumpStart to power Slack AI’s initial search and summarization features and provide safeguards for Slack to use large language models (LLMs) more securely. Slack worked with SageMaker JumpStart to host industry-leading third-party LLMs so that data is not shared with the infrastructure owned by third party model providers. This keeps customer data in Slack at all times and upholds the same security practices and compliance standards that customers expect from Slack itself.

At my work, I'm spending a ton of time reproducing research papers and trying to apply them to our use case (medical image analysis). Usually, that's a big hassle, and even papers-with-code often reveal that it doesn't run without extensive edits. And as soon as they run, results are only good when setting a specific random seed... And then after applying it to our use case, I realized that the performance increase actually doesn't come from a new model, but from a different way of post-processing... I've started to write some scripts for myself to modularize it for easier reproducibility and comparison. But I am not sure if that is the way to go. So I'm curious, what is your approach to reproducing models for your use case and comparing them scientifically? submitted by /u/MintOwlTech [link] [comments]

https://arstechnica.com/tech-policy/2024/04/feds-appoint-ai-doomer-to-run-us-ai-safety-institute/ Article intro: Appointed as head of AI safety is Paul Christiano, a former OpenAI researcher who pioneered a foundational AI safety technique called reinforcement learning from human feedback (RLHF), but is also known for predicting that "there's a 50 percent chance AI development could end in 'doom.'" While Christiano's research background is impressive, some fear that by appointing a so-called "AI doomer," NIST may be risking encouraging non-scientific thinking that many critics view as sheer speculation. submitted by /u/bregav [link] [comments]

In asset management, portfolio managers need to closely monitor companies in their investment universe to identify risks and opportunities, and guide investment decisions. Tracking direct events like earnings reports or credit downgrades is straightforward—you can set up alerts to notify managers of news containing company names. However, detecting second and third-order impacts arising from events

This post walks you through the Open Source Observability pattern for AWS Inferentia, which shows you how to monitor the performance of ML chips, used in an Amazon Elastic Kubernetes Service (Amazon EKS) cluster, with data plane nodes based on Amazon Elastic Compute Cloud (Amazon EC2) instances of type Inf1 and Inf2.

Amazon SageMaker Studio provides a fully managed solution for data scientists to interactively build, train, and deploy machine learning (ML) models. In the process of working on their ML tasks, data scientists typically start their workflow by discovering relevant data sources and connecting to them. They then use SQL to explore, analyze, visualize, and integrate

In this post, we explore the performance benefits of Amazon SageMaker (including SMP and SMDDP), and how you can use the library to train large models efficiently on SageMaker. We demonstrate the performance of SageMaker with benchmarks on ml.p4d.24xlarge clusters up to 128 instances, and FSDP mixed precision with bfloat16 for the Llama 2 model.

Amazon Lex is a fully managed artificial intelligence (AI) service with advanced natural language models to design, build, test, and deploy conversational interfaces in applications. It employs advanced deep learning technologies to understand user input, enabling developers to create chatbots, virtual assistants, and other applications that can interact with users in natural language. Managing your

Generative artificial intelligence (AI) is transforming the customer experience in industries across the globe. Customers are building generative AI applications using large language models (LLMs) and other foundation models (FMs), which enhance customer experiences, transform operations, improve employee productivity, and create new revenue channels. The biggest concern we hear from customers as they explore the advantages of generative AI is how to protect their highly sensitive data and investments. At AWS, our top priority is safeguarding the security and confidentiality of our customers' workloads. We think about security across the three layers of our generative AI stack ...

Organizations across industries want to categorize and extract insights from high volumes of documents of different formats. Manually processing these documents to classify and extract information remains expensive, error prone, and difficult to scale. Advances in generative artificial intelligence (AI) have given rise to intelligent document processing (IDP) solutions that can automate the document classification,

AWS was delighted to present to and connect with over 18,000 in-person and 267,000 virtual attendees at NVIDIA GTC, a global artificial intelligence (AI) conference that took place March 2024 in San Jose, California, returning to a hybrid, in-person experience for the first time since 2019. AWS has had a long-standing collaboration with NVIDIA for

This blog post is co-written with Caroline Chung from Veoneer. Veoneer is a global automotive electronics company and a world leader in automotive electronic safety systems. They offer best-in-class restraint control systems and have delivered over 1 billion electronic control units and crash sensors to car manufacturers globally. The company continues to build on a

With Knowledge Bases for Amazon Bedrock, you can securely connect foundation models (FMs) in Amazon Bedrock to your company data for Retrieval Augmented Generation (RAG). Access to additional data helps the model generate more relevant, context-specific, and accurate responses without retraining the FMs. In this post, we discuss two new features of Knowledge Bases for

At AWS re:Invent 2023, we announced the general availability of Knowledge Bases for Amazon Bedrock. With Knowledge Bases for Amazon Bedrock, you can securely connect foundation models (FMs) in Amazon Bedrock to your company data using a fully managed Retrieval Augmented Generation (RAG) model. For RAG-based applications, the accuracy of the generated responses from FMs

Unlocking accurate and insightful answers from vast amounts of text is an exciting capability enabled by large language models (LLMs). When building LLM applications, it is often necessary to connect and query external data sources to provide relevant context to the model. One popular approach is using Retrieval Augmented Generation (RAG) to create Q&A systems

With the widespread adoption of generative artificial intelligence (AI) solutions, organizations are trying to use these technologies to make their teams more productive. One exciting use case is enabling natural language interactions with relational databases. Rather than writing complex SQL queries, you can describe in plain language what data you want to retrieve or manipulate.

In January 2024, Amazon SageMaker launched a new version (0.26.0) of Large Model Inference (LMI) Deep Learning Containers (DLCs). This version offers support for new models (including Mixture of Experts), performance and usability improvements across inference backends, as well as new generation details for increased control and prediction explainability (such as reason for generation completion

Please post your questions here instead of creating a new thread. Encourage others who create new posts for questions to post here instead! Thread will stay alive until next one so keep posting after the date in the title. Thanks to everyone for answering questions in the previous thread! submitted by /u/AutoModerator [link] [comments]

Amazon Rekognition makes it easy to add image and video analysis to your applications. It’s based on the same proven, highly scalable, deep learning technology developed by Amazon’s computer vision scientists to analyze billions of images and videos daily. It requires no machine learning (ML) expertise to use and we’re continually adding new computer vision

This is a guest post co-authored by Shravan Kumar and Avirat S from Gramener. Gramener, a Straive company, contributes to sustainable development by focusing on agriculture, forestry, water management, and renewable energy. By providing authorities with the tools and insights they need to make informed decisions about environmental and social impact, Gramener is playing a