Database Administrators Stack Exchange
Database Administrators Stack Exchange

Questions tagged [polybase]

Microsoft SQL Server product to allow SQL queries against relational, Hadoop and Azure blob storage.

PolyBase is a technology that accesses and combines both non-relational and relational data, all from within SQL Server. It allows you to run queries on external data in Hadoop or Azure blob storage. The queries are optimized to push computation to Hadoop.

19 questions
6
votes
1 answer

Performance of Polybase

We've been experimenting with Polybase on a SQL Server 2019 (CU2) box, using SQL Server external data sources and the performance has not been good - in most cases a 1400% increase. In each case all of the tables/views we're querying are from…
Select 'DBA'
  • 133
  • 10
5
votes
3 answers

Azure SQL Warehouse - Data Ingestion - Convert a huge fixed width (with commas) file to delimited

I am not even sure if I am framing this question right, but I will try - I have a bunch of huge text files generated from an Oracle export on a Linux system. Each file is about 30 GB in size, and I have about 50 of them. The goal is to export this…
RK Kuppala
  • 2,387
  • 1
  • 20
  • 23
3
votes
1 answer

Can't stop "SQL Server Polybase Data Movement"

When I start Windows the "SQL Server Polybase Data Movement" service is in the "Change pending..." state and I can't close it. When I try to stop it mmc crashes. In Windows services, the status = starting, but it never changes either. I cannot stop…
xenorack
  • 33
  • 1
  • 5
3
votes
1 answer

How does PolyBase collect statistics from external tables?

Books online has this to say about query optimisation in PolyBase: The query optimizer makes a cost-based decision to push computation to Hadoop when doing so will improve query performance. It uses statistics on external tables to make the…
Michael Green
  • 22,481
  • 12
  • 46
  • 87
2
votes
1 answer

SQL Server Polybase Data Movement Service stopping after few seconds

I have been scratching my head for a while now on how to fix this issue, SQL Server Polybase Data Movement Service starts successfully however after a few seconds it just stops. The OS on our server went corrupt last week and force us to reload the…
JPVoogt
  • 223
  • 1
  • 8
2
votes
1 answer

How to ignore columns in parquet/polybase import?

I'm using polybase to import a parquet file. Over time, it is likely we may add or remove named columns in the file. When I add an additional column, I get the below error: External file access failed due to internal error: 'File test.parquet:…
Neil P
  • 1,184
  • 16
  • 33
2
votes
0 answers

PolyBase to CSV File

I'm trying to see if it's possible to create a PolyBase connection to a .CSV file. I'm using the Microsoft Access Text Driver and have set up a system DSN to the folder where the csv file is located. I'm able to query the file using OPENROWSET and…
MrTCS
  • 689
  • 4
  • 12
2
votes
1 answer

Reading decimal mark comma with an external table

SQL Server 2016, SQL Server Management Studio (SSMS). Creating external tables with Transact-SQL and Polybase. How can we read decimals with a decimal mark, comma instead of dot? When using float and decimal, it fails and throws an error. When using…
SNR
  • 139
  • 5
1
vote
0 answers

Polybase converts getdate() to the wrong format

We have two SQL Servers (both running Enterprise version on 15.0.4083.2) where we use queries to External tables via Polybase from one server to the other. We have encountered a strange error when we use the getdate() function to filter against a…
GHauan
  • 553
  • 4
  • 19
1
vote
1 answer

SQL Server 2019 Polybase installation error "The proposed new database owner is already a user or aliased in the database"

I've recently performed an in-place upgrade of the default instance of a SQL Server 2016 to SQL Server 2019 (CU11). I'm trying to add the Polybase feature but am unable to resolve this error that occurs during installation: Error installing SQL…
sTTu
  • 135
  • 9
1
vote
2 answers

Query to polybase external table giving error "Access to the remote server is denied because no login-mapping exists."

we have successfully created all pre-requisite of creating external tables using SQL Server 2017 Standard Edition CU22 Polybase. The issue we have is the non-sysadmin login can't query those external tables .It shows below error. Msg 7416, Level…
Praful
  • 11
  • 1
1
vote
3 answers

Why is Microsoft SQL Server creating a ton of dump files?

How to determine what is causing Microsoft SQL Server to create dump files over and over taking up 170 GB of disk space? How do I troubleshoot what is causing the dumps? The path of the dump files is C:\Program Files\Microsoft SQL…
Tanooki Mario
  • 11
  • 1
  • 2
1
vote
0 answers

How to install PolyBase in an availability groups architecture for high availability?

Background: I’m designing a 2 node SQL 2019 availability groups installation hosted in VMs. We would have limited (single node) usage of PolyBase. The Microsoft Docs don’t reference installation for any HA architectures. It only references how to…
Marshall
  • 21
  • 3
1
vote
2 answers

Missing Polybase enable configuration on SQL Server 2017

I installed SQL Server 2017 Enterprise (14.0.3162.1) on my box. With the polybase (Scale-out Group). But when I try to enable through: exec sp_configure @configname = 'polybase enabled', @configvalue = 1; RECONFIGURE WITH OVERRIDE; It shows: Msg…
DBALUKE HUANG
  • 417
  • 4
  • 15
0
votes
1 answer

SQL Server 2019 Polybase error - cant create External table

I am trying to setup an external data source in SQL Server 2019 to another database on the same server. I am doing this to create external table I tried the following to set it up I installed SQL Server 2019 Enterprise I installed Polybase I…
mohamadreza dehghan
  • 11
  • 4
1
2