For a while I've been looking for a good format to practice implementing ML algorithms. LeetCode feels too disconnected from real work, but in actual projects you just use existing libraries. What worked for me was breaking real algorithms into progressive steps and implementing them piece by piece. I've been using this approach for myself, and recently decided to clean up some of it with tests and hints in case others find it useful. Currently covers: attention, BPE tokenization, beam search variants, and RoPE. Curious if others have found similar formats helpful, or what primitives would be worth adding. submitted by /u/randmusr66 [link] [comments]

So I came across a 2025 KDD paper whose idea is pretty simple and not too novel in my opinion. The paper shared a code link that was broken. But the same paper was rejected from ICLR but had shared the code there. They primarily did experiments on 2 datasets that were public following some training/credentialing steps. I was planning to submit something to KDD this year trying to improve upon this work. I was thinking of simply following their experimental procedure for my method and use the results of all models reported in their paper as baselines. So I emailed the corresponding author who immediately directed the first author to contact me. The first author then shared a Github repo that was created 3 weeks ago. However, the experimental setup was still very vague (like the first preprocessing script assumed that a file is already available while the raw data is spread across directories and there was no clarity about what folders were even used). Initially the author was pretty fast in responding to my emails (took maybe 10-15 mins or so), but as soon as I asked for the script to create this file, they first said that they cannot share the script as the data is behind the credentialing step. However, having worked in this field for 4 years now, I know that you can share codes, but not data in this case. However, I actually sent proof that I have access to the data and shared my data usage agreement. However, it's been 7 hrs or so and no response. I mean, I have seen this type of radio silence from researchers from Chinese Universities before. But the authors of this paper are actually from a good R-1 University in the US. So it was kinda weird. I do not want to specifically reveal the names of the paper or the authors but what is the harm in sharing your experimental setup? I would have actually cited their work had I been able to code this up. Also, I do not get how such a borderline paper (in terms of the technical novelty) with poor reproducibility get into KDD in the first place? submitted by /u/Massive-Bobcat-5363 [link] [comments]

I've been interviewing for research (some engineering) interships for the last 2 months, and I think I'm at a point of mental exhaustion from constant rejections and wasted time. For context, I just started my master’s at Waterloo, but I'm a research associate at one of the top labs in Europe. I have been doing research since my sophomore year. I did not start in ML, but over the last year and a half, I ended up in ML research, first in protein design and now in pretraining optimization. I started applying for interships a few months ago, and after 10+ first-round interviews and endless OAs, I haven't landed any offers. Most of the companies that I've interviewed with were a mix of (non-FAANG) frontier AI companies, established deep tech startups, research labs of F100 companies, a couple non name startups, and a quant firm. I get past a few rounds, then get cut. The feedback in general is that I'm not a good "fit" (a few companies told me I'm too researchy for a research engineer, another few were researching some niche stuff). And the next most common reason is that I failed the coding technical (I have no issue passing the research and ML theory technical interviews), but I think too slow for an engineer, and it's never the same type of questions (with one frontier company, I passed the research but failed the code review) and I'm not even counting OAs. Not a single one asked Leetcode or ML modelling; it's always some sort of a custom task that I have no prior experience with, so it's never the same stuff I can prepare. I'm at a loss, to be honest. Every PhD and a bunch of master's students in our lab have interned at frontier companies, and I feel like a failure that, after so many interviews, I can't get an offer. Because of my CV (no lies), I don't have a problem getting interviews, but I can't seem to get an offer. I've tried applying for non-research and less competitive companies, but I get hit with "not a good fit." I have 3 technicals next week, and tbh I know for a fact I'm not gonna pass 2 of them (too stupid to be a quant researcher) and the other is a 3rd round technical, but from the way he described it I don't think I'll be passing it (they're gonna throw a scientific simulation coding problem at me). And I still need to schedule one more between those 3, but I'm not sure why they even picked me, I don't do RL or robotics research. After so many days and hours spent preparing for each technical only to get cut, I mentally can't get myself to prepare for them anymore. It's always a new random format. I'm severely burned out by this whole process, but time is running out. I love research, but I'm starting to hate the hiring process in this industry. Any advice on what to do? submitted by /u/RNRuben [link] [comments]

Hey everyone! I built vLLM-MLX - a framework that uses Apple's MLX for native GPU acceleration. What it does: - OpenAI-compatible API (drop-in replacement for your existing code) - Multimodal support: Text, Images, Video, Audio - all in one server - Continuous batching for concurrent users (3.4x speedup) - TTS in 10+ languages (Kokoro, Chatterbox models) - MCP tool calling support Performance on M4 Max: - Llama-3.2-1B-4bit → 464 tok/s - Qwen3-0.6B → 402 tok/s - Whisper STT → 197x real-time Works with standard OpenAI Python SDK - just point it to localhost. GitHub: https://github.com/waybarrios/vllm-mlx submitted by /u/waybarrios [link] [comments]

In this post, we show you how fine-tuning enabled a 33% reduction in dangerous medication errors (Amazon Pharmacy), engineering 80% human effort reduction (Amazon Global Engineering Services), and content quality assessments improving 77% to 96% accuracy (Amazon A+). This post details the techniques behind these outcomes: from foundational methods like Supervised Fine-Tuning (SFT) (instruction tuning), and Proximal Policy Optimization (PPO), to Direct Preference Optimization (DPO) for human alignment, to cutting-edge reasoning optimizations such as Grouped-based Reinforcement Learning from Policy Optimization (GRPO), Direct Advantage Policy Optimization (DAPO), and Group Sequence Policy Optimization (GSPO) purpose-built for agentic systems.

Palo Alto Networks’ Device Security team wanted to detect early warning signs of potential production issues to provide more time to SMEs to react to these emerging problems. They partnered with the AWS Generative AI Innovation Center (GenAIIC) to develop an automated log classification pipeline powered by Amazon Bedrock. In this post, we discuss how Amazon Bedrock, through Anthropic’ s Claude Haiku model, and Amazon Titan Text Embeddings work together to automatically classify and analyze log data. We explore how this automated pipeline detects critical issues, examine the solution architecture, and share implementation insights that have delivered measurable operational improvements.

The AWS AI League, launched by Amazon Web Services (AWS), expanded its reach to the Association of Southeast Asian Nations (ASEAN) last year, welcoming student participants from Singapore, Indonesia, Malaysia, Thailand, Vietnam, and the Philippines. In this blog post, you’ll hear directly from the AWS AI League champion, Blix D. Foryasen, as he shares his reflection on the challenges, breakthroughs, and key lessons discovered throughout the competition.

In this post, we demonstrate how to use a GitHub Actions workflow to automate the deployment of AI agents on AgentCore Runtime. This approach delivers a scalable solution with enterprise-level security controls, providing complete continuous integration and delivery (CI/CD) automation.

It looks like ICASSP 2026 decisions may already be accessible. If you can log in to the following link and successfully send an invitation email, that seems to indicate your paper has been accepted: https://cmsworkshops.com/ICASSP2026/author_invitation_request.php The email says: “On behalf of IEEE ICASSP 2026, I invite you to join us for the upcoming conference. We are pleased to inform you that your submission has been accepted for presentation at the 2026 IEEE International Conference on Acoustics, Speech, and Signal Processing (IEEE ICASSP 2026) in Barcelona, Spain, during 3–8 May 2026. ICASSP is the world’s largest and most comprehensive technical conference focused on signal processing and its applications. It offers a comprehensive technical program presenting all the latest development in research and technology in the industry that attracts thousands of professionals annually.” Hopefully this helps others who are anxiously waiting. Good luck everyone Update: It was a bug that got fixed within a few hours. It looks like no one can access it right now. “Error: No match for paper number and password. 0x4C”. submitted by /u/Financial-Panda6581 [link] [comments]

Mamba-2 restructured its recurrence from parallel scans (10-20% Tensor Core utilization) to block-diagonal GEMMs (60-70%). The architecture bent to fit the silicon. RetNet was published by Microsoft Research in July 2023 with promising results at 6.7B. Five months later, the same organization shipped Phi-2, a dense Transformer. Then Phi-3. Then Phi-4. The co-authors didn't bet on their own architecture. I wrote an analysis of why this pattern keeps repeating. The short version: Transformers and NVIDIA GPUs co-evolved into a stable attractor. Breaking out requires clearing two reinforcing gates at once, hardware compatibility and institutional backing, and the gates make each other harder to pass. At frontier scale, no pure alternative has done it. Essay has Tensor Core utilization numbers, analysis of alternative chip vendors, and three falsifiable predictions for 2028. submitted by /u/petroslamb [link] [comments]

I’m looking for some opinions on the use of weight decay in RealNVP-style normalizing flows. My concern is that blindly applying standard weight decay (L2 on parameters) may be actively harmful in this setting. In RealNVP, each coupling layer is explicitly structured so that small weights push the transformation toward the identity map. With weight decay, we’re therefore not just regularizing capacity, we are actually biasing the model towards doing nothing. In flows, the identity transform is a perfectly valid (and often high-likelihood early) solution (especially if you zero init your scale networks which seems to be standard practice), so weight decay feels like it’s reinforcing a bad inductive bias. Most implementations seem to include weight decay by default, but I haven’t seen much discussion about whether it actually makes sense for invertible models. EDIT: Following this post, I took the liberty of exploring this question through a toy problem. The setup is intentionally simple: I train a RealNVP-style flow to map between a standard Gaussian and a learned latent distribution coming from another model I’m working on. The target latent distribution has very small variance (overall std ≈ 0.067, with some dimensions down at 1e-4), which makes the identity-map bias especially relevant. I ran a small ablation comparing no weight decay vs standard L2 (1e-4), keeping everything else fixed. With weight decay 0: === ABLATION CONFIG === weight_decay: 0.0 tanh_scale: 3.0 grad_clip: 1.0 lr: 0.001 epochs: 2000 print_every: 200 Latents: mean=0.0008, std=0.0667 per-dim std: min=0.0002, max=0.1173 === TRAINING === Epoch 200 | NLL: -801.28 | z_std: 0.900 | inv_std: 0.0646 | base1: [0.06573893129825592, 0.04342599958181381, 0.08187682926654816] Epoch 400 | NLL: -865.13 | z_std: 0.848 | inv_std: 0.0611 | base1: [0.10183795541524887, 0.05562306195497513, 0.14103063941001892] Epoch 600 | NLL: -892.77 | z_std: 0.956 | inv_std: 0.0618 | base1: [0.12410587072372437, 0.06660845875740051, 0.1999545693397522] Epoch 800 | NLL: -925.00 | z_std: 1.055 | inv_std: 0.0650 | base1: [0.13949117064476013, 0.07608211040496826, 0.2613525688648224] Epoch 1000 | NLL: -952.22 | z_std: 0.957 | inv_std: 0.0651 | base1: [0.1513708531856537, 0.08401045948266983, 0.3233321011066437] Epoch 1200 | NLL: -962.60 | z_std: 0.930 | inv_std: 0.0630 | base1: [0.16100724041461945, 0.09044866263866425, 0.385517954826355] Epoch 1400 | NLL: -972.35 | z_std: 1.120 | inv_std: 0.0644 | base1: [0.16973918676376343, 0.09588785469532013, 0.4429493546485901] Epoch 1600 | NLL: -1003.05 | z_std: 1.034 | inv_std: 0.0614 | base1: [0.17728091776371002, 0.10034342855215073, 0.4981722831726074] Epoch 1800 | NLL: -1005.57 | z_std: 0.949 | inv_std: 0.0645 | base1: [0.18365693092346191, 0.10299171507358551, 0.5445704460144043] Epoch 2000 | NLL: -1027.24 | z_std: 0.907 | inv_std: 0.0676 | base1: [0.19001561403274536, 0.10608844459056854, 0.5936127305030823] === FINAL EVALUATION === Target: mean=0.0008, std=0.0667 Forward: mean=0.0239, std=0.9074 (should be ~0, ~1) Inverse: mean=0.0009, std=0.0644 (should match target) With weight decay 1e-4: === ABLATION CONFIG === weight_decay: 0.0001 tanh_scale: 3.0 grad_clip: 1.0 lr: 0.001 epochs: 2000 print_every: 200 Latents: mean=0.0008, std=0.0667 per-dim std: min=0.0002, max=0.1173 === TRAINING === Epoch 200 | NLL: -766.17 | z_std: 0.813 | inv_std: 0.1576 | base1: [0.06523454189300537, 0.04702048376202583, 0.07113225013017654] Epoch 400 | NLL: -795.67 | z_std: 1.064 | inv_std: 0.7390 | base1: [0.08956282585859299, 0.0620030015707016, 0.10142181813716888] Epoch 600 | NLL: -786.70 | z_std: 1.004 | inv_std: 0.1259 | base1: [0.09346793591976166, 0.06835056096315384, 0.11534363776445389] Epoch 800 | NLL: -772.45 | z_std: 1.146 | inv_std: 0.1531 | base1: [0.09313802421092987, 0.06970944255590439, 0.12027867138385773] Epoch 1000 | NLL: -825.67 | z_std: 0.747 | inv_std: 0.1728 | base1: [0.09319467097520828, 0.06899876147508621, 0.12167126685380936] Epoch 1200 | NLL: -817.38 | z_std: 0.911 | inv_std: 0.1780 | base1: [0.09275200963020325, 0.06717729568481445, 0.12130238860845566] Epoch 1400 | NLL: -831.18 | z_std: 0.722 | inv_std: 0.1677 | base1: [0.0924605205655098, 0.0654158964753151, 0.1201595664024353] Epoch 1600 | NLL: -833.45 | z_std: 0.889 | inv_std: 0.1919 | base1: [0.09225902706384659, 0.06358200311660767, 0.11815735697746277] Epoch 1800 | NLL: -838.98 | z_std: 0.893 | inv_std: 0.1714 | base1: [0.09210160374641418, 0.06210005283355713, 0.11663311719894409] Epoch 2000 | NLL: -832.70 | z_std: 0.812 | inv_std: 0.1860 | base1: [0.0919715166091919, 0.060423776507377625, 0.11383745074272156] === FINAL EVALUATION === Target: mean=0.0008, std=0.0667 Forward: mean=-0.0090, std=0.8116 (should be ~0, ~1) Inverse: mean=0.0023, std=0.2111 (should match target) Without weight decay, the model steadily moves away from the identity. The inverse pass closely matches the target latent statistics, and the forward pass converges to something very close to a standard normal (std ≈ 0.91 by the end, still improving). NLL improves monotonically, and the learned base transform parameters keep growing, indicating the model is actually using its capacity. With weight decay, training is noticeably different. NLL plateaus much earlier and fluctuates. More importantly, the inverse mapping never fully contracts to the target latent distribution (final inverse std ≈ 0.21 vs target 0.067). The forward mapping also under-disperses (std ≈ 0.81). Qualitatively, this looks exactly like the concern I raised originally: weight decay doesn’t just regularize complexity here. Now, I’m not claiming this means “never use weight decay in flows,” but in appears that indeed in certain settings one should definitely think twice :D. submitted by /u/Screech-1 [link] [comments]

We’ve been doing sentiment analysis on text forever(tweets, reviews, comments, etc). But what about video? With so much content now being video-first (YouTube, TikTok, ads, UGC, webinars), I’m wondering if anyone is actually doing sentiment analysis on video in a serious way. Things like: detecting positive / negative tone in spoken video understanding context around product mentions knowing when something is said in a video, not just that it was said analysing long videos, not just short clips I’m curious if: this is already being used in the real world it’s mostly research / experimental or people still just rely on transcripts + basic metrics Would love to hear from anyone in ML, data, marketing analytics, or CV who’s seen this in practice or experiemented with it. submitted by /u/YiannisPits91 [link] [comments]

Zhipu AI and Huawei just dropped GLM-Image, and the technical details are interesting. First multimodal model trained completely on Chinese chips (Huawei Ascend 910) from data preprocessing to full scale training. They're using a hybrid architecture combining autoregressive + diffusion decoder. What stands out is the Chinese text rendering. It consistently ranks first among open source models for complex text generation, especially handling Chinese characters which most models struggle with. Native support for 1024 to 2048 resolution at any aspect ratio without additional training. API pricing is 0.1 yuan per image (roughly $0.014). The model handles both text to image and image to image generation in a single model. GitHub and Hugging Face repos are already up. This is significant because it proves you can train frontier models without relying on Nvidia hardware. The compute efficiency numbers they're claiming are 60% better than H200 for tokens per joule. Whether those benchmarks hold up in practice remains to be seen but the fact they pulled this off on domestic hardware is noteworthy. submitted by /u/Different_Case_6484 [link] [comments]

To all DS and ML researchers If someone got tired of copy-pasting the same data loading, training loops, and export code for every CV project. So I built a toolkit that handles the boring stuff. What it does: from cv_pipeline import quick_train, analyze_dataset, export_model # Analyze your dataset analyze_dataset("./my_images") # Train (one line) model, history = quick_train("./my_images", model="efficientnet_b0", epochs=10) # Export for deployment export_model(model, "model.onnx", format="onnx") Key features: Data loading - Point to a folder, get DataLoaders. Handles splits, augmentation, and normalisation. 50+ architectures - ResNet, EfficientNet, ViT, MobileNet via timm. One-line model loading. Dataset analysis - Class distribution, imbalance detection, image stats. Model comparison: benchmark multiple architectures on your data. Export - TorchScript, ONNX, state_dict. CLI - cv-pipeline train --data ./images --model resnet50 --epochs 20 Notebook generator - Auto-generate starter notebooks for classification/detection/segmentation. CLI example: # Analyze dataset cv-pipeline analyze --data ./images # Train cv-pipeline train --data ./images --model efficientnet_b0 --epochs 20 # Compare models cv-pipeline compare --models resnet50,efficientnet_b0,vit_base --data ./images Not a framework - just utilities. Use with your existing PyTorch code. No lock-in. Built for rapid prototyping and experiment iteration. Includes configs for medical imaging, manufacturing QC, retail, and document processing use cases. GitHub: https://github.com/var1914/pytorch-ml-pipeline Feedback welcome. What utilities would you add? submitted by /u/Extension_Key_5970 [link] [comments]

I recently came across the Google Scholar profile of a high school student and was quite astonished by the strength of his publication record. Even more strikingly, he is also serving as a reviewer for ICLR and AISTATS. submitted by /u/ApprehensiveEgg5201 [link] [comments]

Hi, I'm looking for help into preparing for the upcoming coding interviews for an ML research engineer position I applied to at Scale. These are for the onsite. The first coding question relates parsing data, data transformations, getting statistics about the data. The second (ML) coding involves ML concepts, LLMs, and debugging. I found the description of the ML part to be a bit vague. For those that have done this type of interview, what did you do to prepare? So far on my list, I have reviewing hyperparameters of LLMs, PyTorch debugging, transformer debugging, and data pipeline pre-processing, ingestion, etc. Will I need to implement NLP or CV algorithms from scratch? Any insight to this would be really helpful. submitted by /u/sailor-goon-is-here [link] [comments]

I am an open source contributor, working on load balancing for Bifrost (LLM gateway) and ran into some interesting challenges with Go implementation. Standard weighted round-robin works fine for static loads, but LLM providers behave weirdly. OpenAI might be fast at 9am, slow at 2pm. Azure rate limits kick in unexpectedly. One region degrades while others stay healthy. Built adaptive routing that adjusts weights based on live metrics - latency, error rates, throughput. Used EWMAs (exponentially weighted moving averages) to smooth out spikes without overreacting to noise. The Go part that was tricky: tracking per-provider metrics without locks becoming a bottleneck at high RPS. Ended up using atomic operations for counters and a separate goroutine that periodically reads metrics and recalculates weights. Keeps the hot path lock-free. Also had to handle provider health scoring. Not just "up or down" but scoring based on recent performance. A provider recovering from issues should gradually earn traffic back, not get slammed immediately. Connection pooling matters more than expected. Go's http.Transport reuses connections well, but tuning MaxIdleConnsPerHost made a noticeable difference under sustained load. Running this at 5K RPS with sub-microsecond overhead now. The concurrency primitives in Go made this way easier than Python would've been. Anyone else built adaptive routing in Go? What patterns worked for you? submitted by /u/dinkinflika0 [link] [comments]

In this post, we explain how we overcame the limitations of single-agent AI systems through a human-centric approach, implemented structured outputs to significantly reduce hallucinations and built a scalable solution now positioned for expansion across the AMET QA team and later across other QA teams in International Emerging Stores and Payments (IESP) Org.

This post introduces generative AI guided business reporting—with a focus on writing achievements & challenges about your business—providing a smart, practical solution that helps simplify and accelerate internal communication and reporting.

In this post, we demonstrate how you can address these challenges by adding centralized safeguards to a custom multi-provider generative AI gateway using Amazon Bedrock Guardrails.

In this post, we describe how you can use Amazon Nova Multimodal Embeddings to retrieve specific video segments. We also review a real-world use case in which Nova Multimodal Embeddings achieved a recall success rate of 96.7% and a high-precision recall of 73.3% (returning the target content in the top two results) when tested against a library of 170 gaming creative assets. The model also demonstrates strong cross-language capabilities with minimal performance degradation across multiple languages.

Hi everyone! Please bear with me with this question 🫣 I’m looking for someone in research to pick their brain about the similarities and differences between statistical learning in cognitive science and in machine learning, so definition, conceptual differences/similarities, predictions, testing…. Hope it makes sense, I’m doing research in cognitive sciences and I’d love to learn more about this term’s use in ML for a review I’m working on 🙂 thanks! submitted by /u/Ok_Fudge1993 [link] [comments]

Just read through this new systematic review (arXiv:2601.09334) on Serverless for HPC/AI. It’s a solid read if you're dealing with infrastructure scaling. The TL;DR: Static Allocation is breaking: The paper argues that rigid GPU clusters can't handle modern "bursty" AI workloads efficiently. You either over-provision (waste money) or under-provision (crash during spikes). Serverless is the fix: The industry is moving toward elastic, serverless execution models to survive the efficiency gap. We've been seeing this exact pattern in production. We actually built our engine specifically to solve that Cold Start problem via state snapshotting, so it's validating to see the academic side converging on the same architecture. Paper link: https://arxiv.org/abs/2601.09334 Anyone seeing this shift from static -> serverless in their own clusters? submitted by /u/pmv143 [link] [comments]

Results out for ISBI 2026 - London a few days back. Just want to check with fellow medical imaging peeps on how did it go for all. Results were delayed by a month and I see a pretty high acceptance rate this time. submitted by /u/ade17_in [link] [comments]

TL;DR: The paper describes a mechanism that essentially turns the context window into a training dataset for a "fast weight" update loop: Inner Loop: The model runs a mini-gradient descent on the context during inference. It updates specific MLP layers to "learn" the current context. Outer Loop: The model's initial weights are meta-learned during training to be "highly updateable" or optimized for this test-time adaptation From the Paper: "Overall, our empirical observations strongly indicate that TTT-E2E should produce the same trend as full attention for scaling with training compute in large-budget production runs." Abstract: We formulate long-context language modeling as a problem in continual learning rather than architecture design. Under this formulation, we only use a standard architecture a Transformer with sliding-window attention. However, our model continues learning at test time via next-token prediction on the given context, compressing the context it reads into its weights. In addition, we improve the model's initialization for learning at test time via meta-learning at training time. Overall, our method, a form of Test-Time Training (TTT), is End-to-End (E2E) both at test time (via next-token prediction) and training time (via meta-learning), in contrast to previous forms. We conduct extensive experiments with a focus on scaling properties. In particular, for 3B models trained with 164B tokens, our method (TTT-E2E) scales with context length in the same way as Transformer with full attention, while others, such as Mamba 2 and Gated DeltaNet, do not. However, similar to RNNs, TTT-E2E has constant inference latency regardless of context length, making it 2.7x faster than full attention for 128K context. Our code is publicly available. Layman's Explanation: Think of this paper as solving the memory bottleneck by fundamentally changing how a model processes information. Imagine you are taking a massive open-book exam. A standard Transformer (like GPT-4) is the student who frantically re-reads every single page of the textbook before answering every single question. This strategy guarantees they find the specific details (perfect recall), but as the textbook gets thicker, they get exponentially slower until they simply cannot finish the test in time. On the other hand, alternatives like RNNs or Mamba try to summarize the entire textbook onto a single index card. They can answer questions instantly because they don't have to look back at the book, but for long, complex subjects, they eventually run out of space on the card and start forgetting crucial information. This new method, Test-Time Training (TTT), changes the paradigm from retrieving information to learning it on the fly. Instead of re-reading the book or summarizing it onto a card, the TTT model treats the context window as a dataset and actually trains itself on it in real-time. It performs a mini-gradient descent update on its own neural weights as it reads. This is equivalent to a student who reads the textbook and physically rewires their brain to master the subject matter before the test. Because the information is now compressed into the model's actual intelligence (its weights) rather than a temporary cache, the model can answer questions instantly (matching the constant speed of the fast index-card models) but with the high accuracy and scaling capability of the slow, page-turning Transformers. This effectively decouples intelligence from memory costs, allowing for massive context lengths without the usual slowdown. Link to the Paper: https://arxiv.org/pdf/2512.23675 Link to the Open-Sourced Official Implementation of End-to-End Test Time Training for Long Context: https://github.com/test-time-training/e2e submitted by /u/44th--Hokage [link] [comments]

In this post, we explore the architecture that AutoScout24 used to build their standardized AI development framework, enabling rapid deployment of secure and scalable AI agents.

This post explores how new serverless model customization capabilities, elastic training, checkpointless training, and serverless MLflow work together to accelerate your AI development from months to days.

I Work on Bifrost (been posting a lot here lol) and wanted to share what we learned building multi-provider routing, since it's messier than it seems. Github: https://github.com/maximhq/bifrost Initially thought weighted routing would be the main thing - like send 80% of traffic to Azure, 20% to OpenAI. Pretty straightforward. Configure weights, distribute requests proportionally, done. But production is messier. Providers go down regionally. Rate limits hit unexpectedly. Azure might be healthy in US-East but degraded in EU-West. Or you hit your tier limit mid-day and everything starts timing out. So we built automatic fallback chains. When you configure multiple providers on a virtual key, Bifrost sorts them by weight and creates fallbacks automatically. Primary request goes to Azure, fails, immediately retries with OpenAI. Happens transparently - your app doesn't see it. The health monitoring part was interesting. We track success rates, response times, error patterns per provider. When issues get detected, requests start routing to backup providers within milliseconds. No manual intervention needed. Also handles rate limits differently now. If a provider hits TPM/RPM limits, it gets excluded from routing temporarily while other providers stay available. Prevents cascading failures. One thing that surprised us - weighted routing alone isn't enough. You need adaptive load balancing that actually looks at real-time metrics (latency, error rates, throughput) and adjusts on the fly. Static weights don't account for degradation. The tricky part was making failover fast enough that it doesn't add noticeable latency. Had to optimize connection pooling, timeout handling, and how we track provider health. how are you folks handling multi-provider routing in production. Static configs? Manual switching? Something else? submitted by /u/dinkinflika0 [link] [comments]

Hi everyone I’m a fellowship-trained neurosurgeon / spine surgeon. I’ve been discussing a persistent problem in our field with other surgeons for a while, and I wanted to run it by people who think about ML systems, not just model performance. I’m trying to pressure-test whether a particular approach is even technically sound, where it would break, and what I’m likely underestimating. Id love to find an interested person to have a discussion with to get a 10000 feet level understanding of the scope of what I am trying to accomplish. The clinical problem: For the same spine pathology and very similar patient presentations, you can see multiple reputable surgeons and get very different surgical recommendations. anything from continued conservative management to decompression, short fusion, or long multilevel constructs. Costs and outcomes vary widely. This isn’t because surgeons are careless. It’s because spine surgery operates with: Limited prospective evidence Inconsistent documentation Weak outcome feedback loops Retrospective datasets that are biased, incomplete, and poorly labeled EMRs are essentially digital paper charts. PACS is built for viewing images, not capturing decision intent. Surgical reasoning is visual, spatial, and 3D, yet we reduce it to free-text notes after the fact. From a data perspective, the learning signal is pretty broken. Why I’m skeptical that training on existing data works: “Labels” are often inferred indirectly (billing codes, op notes) Surgeon decision policies are non-stationary Available datasets are institution-specific and access-restricted Selection bias is extreme (who gets surgery vs who doesn’t is itself a learned policy) Outcomes are delayed, noisy, and confounded Even with access, I’m not convinced retrospective supervision converges to something clinically useful. The idea I’m exploring: Instead of trying to clean bad data later, what if the workflow itself generated structured, high-fidelity labels as a byproduct of doing the work, or at least the majority of it? Concretely, I’m imagining an EMR-adjacent, spine-specific surgical planning and case monitoring environment that surgeons would actually want to use. Not another PACS viewer, but a system that allows: 3D reconstruction from pre-op imaging Automated calculation of alignment parameters Explicit marking of anatomic features tied to symptoms Surgical plan modeling (levels, implants, trajectories, correction goals) Structured logging of surgical cases (to derive patterns and analyze for trends) Enable productivity (generate note, auto populate plans ect.) Enable standardized automated patient outcomes data collection. The key point isn’t the UI, but UI is also an area that currently suffers. It’s that surgeons would be forced (in a useful way) to externalize decision intent in a structured format because it directly helps them plan cases and generate documentation. Labeling wouldn’t feel like labeling it would almost just be how you work. The data used for learning would explicitly include post-operative outcomes. PROMs collected at standardized intervals, complications (SSI, reoperation), operative time, etc, with automated follow-up built into the system. The goal would not be to replicate surgeon decisions, but to learn decision patterns that are associated with better outcomes. Surgeons could specify what they want to optimize for a given patient (eg pain relief vs complication risk vs durability), and the system would generate predictions conditioned on those objectives. Over time, this would generate: Surgeon-specific decision + outcome datasets Aggregate cross-surgeon data Explicit representations of surgical choices, not just endpoints Learning systems could then train on: Individual surgeon decision–outcome mappings Population-level patterns Areas of divergence where similar cases lead to different choices and outcomes Where I’m unsure, and why I’m posting here: From an ML perspective, I’m trying to understand: Given delayed, noisy outcomes, is this best framed as supervised prediction or closer to learning decision policies under uncertainty? How feasible is it to attribute outcome differences to surgical decisions rather than execution, environment, or case selection? Does it make sense to learn surgeon-specific decision–outcome mappings before attempting cross-surgeon generalization? How would you prevent optimizing for measurable metrics (PROMs, SSI, etc) at the expense of unmeasured but important patient outcomes? Which outcome signals are realistically usable for learning, and which are too delayed or confounded? What failure modes jump out immediately? I’m also trying to get a realistic sense of: The data engineering complexity this implies Rough scale of compute once models actually exist The kind of team required to even attempt this (beyond just training models) I know there are a lot of missing details. If anyone here has worked on complex ML systems tightly coupled to real-world workflows (medical imaging, decision support, etc) and finds this interesting, I’d love to continue the discussion privately or over Zoom. Maybe we can collaborate on some level! Appreciate any critique especially the uncomfortable kind!! submitted by /u/LaniakeaResident [link] [comments]

Methodology: 10 frontier models (Claude Opus/Sonnet 4.5, o1, GPT-4o, Gemini 3 Pro, Grok 4, DeepSeek V3.2, Llama 4 Scout, Mistral Large, Command A) Each answers identical prompt blindly All 10 judge all 10 responses (100 judgments) Self-judgments excluded from final scores 5 criteria: Correctness (30%), Completeness (20%), Clarity (20%), Depth (15%), Usefulness (15%) CODE-001 Results (Async Python Debugging): Claude Opus 4.5: 9.49 o1: 9.48 Claude Sonnet 4.5: 9.41 DeepSeek V3.2: 9.39 Grok 4: 9.37 Command A: 9.23 Gemini 3 Pro: 9.19 Mistral Large: 9.10 GPT-4o: 8.79 Llama 4 Scout: 8.04 REASON-001 Results (Two Envelope Paradox): Claude Opus 4.5: 9.24 o1: 9.23 Claude Sonnet 4.5: 9.09 DeepSeek V3.2: 8.93 Grok 4: 8.88 GPT-4o: 8.75 Gemini 3 Pro: 8.68 Mistral Large: 8.64 Command A: 8.38 Llama 4 Scout: 7.92 Judge Bias Patterns: Strictest: Claude Opus (avg 7.10-8.76 depending on task) Most lenient: Mistral Large (9.22-9.73) Correlation: Strict judges tend to score higher themselves Open questions for feedback: Is 5-point rubric weighting optimal for different task types? Should we normalize for judge harshness before aggregating? Are 9 judgments per response sufficient for statistical validity? Full data + prompts: https://themultivac.substack.com Daily evals at themultivac.com — currently in Phase 2 (peer matrix format). submitted by /u/Silver_Raspberry_811 [link] [comments]

I know most will wonder why I’m wasting my time training at only 19k tok a sec. It’s because I can. I’m doing this in my living room in my spare time. 0 formal ML experience. The absurd amount I’ve learned in the last few months made me realize I really picked the wrong career. My Mixture of Experts is 2.36B parameter with 8 routed experts plus a shared expert using top-2 routing. Attention is Grouped Query Attention with QK-normalization and RoPE positional embeddings. All feed-forward layers use SwiGLU activation with RMSNorm throughout. Load balancing follows DeepSeek V3’s auxiliary-loss-free approach using bias-based routing. I monitor coefficient of variation and maximum violation per step. Training runs on TorchAO FP8 quantization with the Muon optimizer and a multi-stage learning rate schedule (warmup, constant, cosine decay). The backend is optimized for Blackwell architecture with cuBLASLt. The data pipeline implements MeCo (Metadata Conditioning then Cooldown) with ledger-based deterministic sampling. I have document-aware attention masking and cross-document loss masking but was disabled for the initial MeCo run. I have since disabled MeCo and curated a clean corpus with no tagging of any kind. MeCo worked but it worked too well and with only 8 experts, it became very problematic. My two biggest early mistakes were not using symmetric router initialization (std=0.006) and not having a dense first layer. Cost me a lot of time and sleep. So what did I do? I cheated. I used aux loss of .003 snd ema smoothing at the beginning. I just didn’t know better. I paid a price later on for that. DO NOT use router scaling on a small MoE. DeepSeek used 2.5. Kimi K2 used 2.446. I tried 1.2 and it was horribly unstable and violation blew up to over .500. 24 batch 6 Grad LR 3e-4 AdamW+Muon Scaled. Bias .001 Aux .0001. I update every step. As of yesterday: 2026-01-13 20:53:06 step 41915 | lr 3.00e-04 | loss 1.8867 | gnorm 0.13 | 19,415 tok/s (ema 19,553) | 75.9s/5 steps | cv 0.022 | bias -0.001708±0.179996 | rel_max=0.036 maxvio=0.027 ent=1.203 applied=True | seq_aux 2.444 2026-01-13 20:54:20 [moe] token counts: [150018, 148422, 155402, 147966, 145236, 146724, 144358, 141522] 2026-01-13 20:54:20 step 41920 | lr 3.00e-04 | loss 1.9263 | gnorm 0.13 | 20,102 tok/s (ema 19,828) | 73.4s/5 steps | cv 0.026 | bias -0.001708±0.179920 | rel_max=0.054 maxvio=0.054 ent=1.211 applied=True | seq_aux 2.515 I got a long ways to go 🙂 I’ll gladly answer any question. No gate keeping here. submitted by /u/exhorder72 [link] [comments]

TL;DR: The paper introduces Spectral Sphere Optimizer, which takes steepest descent under spectral norm (Muon) and forces the weights & updates onto a spectral sphere. Paper: https://www.arxiv.org/pdf/2601.08393 Repo: https://github.com/Unakar/Spectral-Sphere-Optimizer Abstract: Scaling large models requires optimization strategies that ensure rapid convergence grounded in stability. Maximal Update Parametrization ( muP) provides a theoretical safeguard for width-invariant theta(1) activation control, whereas emerging optimizers like Muon are only ``half-aligned'' with these constraints: they control updates but allow weights to drift. To address this limitation, we introduce the Spectral Sphere Optimizer (SSO), which enforces strict module-wise spectral constraints on both weights and their updates. By deriving the steepest descent direction on the spectral sphere, SSO realizes a fully muP-aligned optimization process. To enable large-scale training, we implement SSO as an efficient parallel algorithm within Megatron. Through extensive pretraining on diverse architectures, including Dense 1.7B, MoE 8B-A1B, and 200-layer DeepNet models, SSO consistently outperforms AdamW and Muon. Furthermore, we observe significant practical stability benefits, including improved MoE router load balancing, suppressed outliers, and strictly bounded activations. Algorithm: https://preview.redd.it/f1bvi7yd1cdg1.png?width=1197&format=png&auto=webp&s=88a15a375316f54b092e8101e492a2574dc2ace1 Evals: https://preview.redd.it/5hefuy7g1cdg1.png?width=1503&format=png&auto=webp&s=8a0864c5279654a1c9a29b7aae57d2a1b160aa4d https://preview.redd.it/0sy8ih8h1cdg1.png?width=1517&format=png&auto=webp&s=ffd675a60192908ed95652b89540cce8d2110088 https://preview.redd.it/rz6bhc6i1cdg1.png?width=1585&format=png&auto=webp&s=50cd471c7805517d0279877fee235dea3e42954e https://preview.redd.it/fu5wd7zi1cdg1.png?width=1524&format=png&auto=webp&s=5bfb7668a76ceefa320d7325b6abdb731d985e45 submitted by /u/StartledWatermelon [link] [comments]

Hi everyone, I’m trying to make a deliberate choice between two paths for machine learning and AI development, and I’d really value input from people who’ve used both CUDA GPUs and Apple Silicon. Context I already own a MacBook Pro M1, which I use daily for coding and general work. I’m now considering adding a local CUDA workstation mainly for: Local LLM inference (30B–70B models) Real-time AI projects (LLM + TTS + RVC) Unreal Engine 5 + AI-driven characters ML experimentation and systems-level learning I’m also thinking long-term about portfolio quality and employability (FAANG / ML infra / quant-style roles). Option A — Apple Silicon–first Stick with the M1 MacBook Pro Use Metal / MPS where possible Offload heavy jobs to cloud GPUs (AWS, etc.) Pros I see: efficiency, quiet, great dev experience Concerns: lack of CUDA, tooling gaps, transferability to industry infra Option B — Local CUDA workstation Used build (~£1,270 / ~$1,700): RTX 3090 (24GB) i5-13600K 32GB DDR4 (upgradeable) Pros I see: CUDA ecosystem, local latency, hands-on GPU systems work Concerns: power, noise, cost, maintenance What I’d love feedback on For local LLMs and real-time pipelines, how limiting is Apple Silicon today vs CUDA? For those who’ve used both, where did Apple Silicon shine — and where did it fall short? From a portfolio / hiring perspective, does CUDA experience meaningfully matter in practice? Is a local 3090 still a solid learning platform in 2025, or is cloud-first the smarter move? Is the build I found a good deal ? I’m not anti-Mac (I use one daily), but I want to be realistic about what builds strong, credible ML experience. Thanks in advance — especially interested in responses from people who’ve run real workloads on both platforms. submitted by /u/Individual-School-07 [link] [comments]

In this post, we explore the security considerations and best practices for implementing Amazon Bedrock cross-Region inference profiles. Whether you're building a generative AI application or need to meet specific regional compliance requirements, this guide will help you understand the secure architecture of Amazon Bedrock CRIS and how to properly configure your implementation.

This post is co-written with Sunaina Kavi, AI/ML Product Manager at Omada Health. Omada Health, a longtime innovator in virtual healthcare delivery, launched a new nutrition experience in 2025, featuring OmadaSpark, an AI agent trained with robust clinical input that delivers real-time motivational interviewing and nutrition education. It was built on AWS. OmadaSpark was designed

In this post, we explore how Amazon Nova Multimodal Embeddings addresses the challenges of crossmodal search through a practical ecommerce use case. We examine the technical limitations of traditional approaches and demonstrate how Amazon Nova Multimodal Embeddings enables retrieval across text, images, and other modalities. You learn how to implement a crossmodal search system by generating embeddings, handling queries, and measuring performance. We provide working code examples and share how to add these capabilities to your applications.

Quantized models can be seamlessly deployed on Amazon SageMaker AI using a few lines of code. In this post, we explore why quantization matters—how it enables lower-cost inference, supports deployment on resource-constrained hardware, and reduces both the financial and environmental impact of modern LLMs, while preserving most of their original performance. We also take a deep dive into the principles behind PTQ and demonstrate how to quantize the model of your choice and deploy it on Amazon SageMaker.

Beekeeper’s automated leaderboard approach and human feedback loop system for dynamic LLM and prompt pair selection addresses the key challenges organizations face in navigating the rapidly evolving landscape of language models.

This post, developed through a strategic scientific partnership between AWS and the Instituto de Ciência e Tecnologia Itaú (ICTi), P&D hub maintained by Itaú Unibanco, the largest private bank in Latin America, explores the technical aspects of sentiment analysis for both text and audio. We present experiments comparing multiple machine learning (ML) models and services, discuss the trade-offs and pitfalls of each approach, and highlight how AWS services can be orchestrated to build robust, end-to-end solutions. We also offer insights into potential future directions, including more advanced prompt engineering for large language models (LLMs) and expanding the scope of audio-based analysis to capture emotional cues that text data alone might miss.

This post provides a detailed architectural overview of how TrueLook built its AI-powered safety monitoring system using SageMaker AI, highlighting key technical decisions, pipeline design patterns, and MLOps best practices. You will gain valuable insights into designing scalable computer vision solutions on AWS, particularly around model training workflows, automated pipeline creation, and production deployment strategies for real-time inference.

This two-part series explores Flo Health's journey with generative AI for medical content verification. Part 1 examines our proof of concept (PoC), including the initial solution, capabilities, and early results. Part 2 covers focusing on scaling challenges and real-world implementation. Each article stands alone while collectively showing how AI transforms medical content management at scale.

This post shows an automated PII detection and redaction solution using Amazon Bedrock Data Automation and Amazon Bedrock Guardrails through a use case of processing text and image content in high volumes of incoming emails and attachments. The solution features a complete email processing workflow with a React-based user interface for authorized personnel to more securely manage and review redacted email communications and attachments. We walk through the step-by-step solution implementation procedures used to deploy this solution. Finally, we discuss the solution benefits, including operational efficiency, scalability, security and compliance, and adaptability.

Observe.ai developed the One Load Audit Framework (OLAF), which integrates with SageMaker to identify bottlenecks and performance issues in ML services, offering latency and throughput measurements under both static and dynamic data loads. In this blog post, you will learn how to use the OLAF utility to test and validate your SageMaker endpoint.

Please post your personal projects, startups, product placements, collaboration needs, blogs etc. Please mention the payment and pricing requirements for products and services. Please do not post link shorteners, link aggregator websites , or auto-subscribe links. -- Any abuse of trust will lead to bans. Encourage others who create new posts for questions to post here instead! Thread will stay alive until next one so keep posting after the date in the title. -- Meta: This is an experiment. If the community doesnt like this, we will cancel it. This is to encourage those in the community to promote their work by not spamming the main threads. submitted by /u/AutoModerator [link] [comments]

This article is divided into five parts; they are: • An Example of Tensor Parallelism • Setting Up Tensor Parallelism • Preparing Model for Tensor Parallelism • Train a Model with Tensor Parallelism • Combining Tensor Parallelism with FSDP Tensor parallelism originated from the Megatron-LM paper.

For Job Postings please use this template Hiring: [Location], Salary:[], [Remote | Relocation], [Full Time | Contract | Part Time] and [Brief overview, what you're looking for] For Those looking for jobs please use this template Want to be Hired: [Location], Salary Expectation:[], [Remote | Relocation], [Full Time | Contract | Part Time] Resume: [Link to resume] and [Brief overview, what you're looking for] ​ Please remember that this community is geared towards those with experience. submitted by /u/AutoModerator [link] [comments]

This article is divided into five parts; they are: • Introduction to Fully Sharded Data Parallel • Preparing Model for FSDP Training • Training Loop with FSDP • Fine-Tuning FSDP Behavior • Checkpointing FSDP Models Sharding is a term originally used in database management systems, where it refers to dividing a database into smaller units, called shards, to improve performance.

If you've built chatbots or worked with language models, you're already familiar with how AI systems handle memory within a single conversation.

This article is divided into six parts; they are: • Pipeline Parallelism Overview • Model Preparation for Pipeline Parallelism • Stage and Pipeline Schedule • Training Loop • Distributed Checkpointing • Limitations of Pipeline Parallelism Pipeline parallelism means creating the model as a pipeline of stages.

Predicting the future has always been the holy grail of analytics.

This article is divided into two parts; they are: • Data Parallelism • Distributed Data Parallelism If you have multiple GPUs, you can combine them to operate as a single GPU with greater memory capacity.

This article is divided into two parts; they are: • Using `torch.

This article is divided into three parts; they are: • Floating-point Numbers • Automatic Mixed Precision Training • Gradient Checkpointing Let's get started! The default data type in PyTorch is the IEEE 754 32-bit floating-point format, also known as single precision.

If you have an interest in agentic coding, there's a pretty good chance you've heard of

This article is divided into two parts; they are: • What Is Perplexity and How to Compute It • Evaluate the Perplexity of a Language Model with HellaSwag Dataset Perplexity is a measure of how well a language model predicts a sample of text.